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Executive  Summary 


Background.  The  remediation  of  sites  contaminated  with  unexploded  ordnance  (UXO) 
remains  an  area  of  intense  focus  for  the  Department  of  Defense.  Current  estimates  place 
the  total  area  of  possibly  UXO-contaminated  sites  at  10  million  acres,  with  an  overall 
cost  of  remediation  with  current  methods  and  sensing  technologies  in  the  tens  of  billions 
of  dollars.  Fortunately,  studies  have  estimated  that  up  to  80%  of  typical  sites  of  potential 
contamination  are  actually  UXO-free.  What  is  needed  to  take  advantage  of  this  ratio  is  a 
means  to  quickly  and  reliably  scan  large  sites  (on  the  order  of  10,000  acres)  in  order  to 
rapidly  identify  regions  that  are  free  of  UXO  and  regions  that  must  be  subjected  to  more 
detailed  and  time-intensive  examination  and  remediation  with  established  UXO  detection 
tools.  Recent  investigations  have  focused  on  wide-area  assessments  (WAA)  aimed  at 
rapidly  determining  the  approximate  density  and  spatial  distribution  of  UXO  objects  over 
regions  of  wide  area,  rather  than  identification  of  individual  UXO  objects.  Six  wide-area 
assessment  projects  have  been  completed  under  the  auspices  of  Strategic  Environmental 
Research  and  Development  Program  (SERDP)  and  Environmental  Security  Technology 
Certification  Program  (ESTCP).  [1]  These  projects  utilized  various  detection  techniques 
with  different  strengths  and  weaknesses.  However,  effective  wide-area  assessment  has 
been  hindered  by  a  lack  of  accurate  target  and  range  information.  In  addition,  no  single 
sensing  technology  has  been  both  accurate  and  cost-effective  in  surveying  entire  sites. 
Therefore,  this  project  will  work  closely  with  the  ESTCP  Wide-Area  Assessment  Pilot 
Program  to  examine  intelligent  data  fusion  to  improve  detection  and  identification. 

Objective.  Intelligent  data  fusion  techniques  are  being  developed  and  optimized  for  use 
in  enhancing  wide-area  assessment  UXO  remediation  efforts.  A  data  fusion  framework 
will  be  created  to  provide  a  cohesive  data  management  and  decision  making  utility  that 
will  capture  all  available  data  and  more  efficiently  direct  the  expenditure  of  time,  labor 
and  resources.  In  the  first  year  of  this  project,  the  objectives  are  to  determine  the 
feasibility  of  feature  selection  methods  for  data  fusion. 

Methods.  In  the  context  of  UXO  remediation,  intelligent  data  fusion  is  the  combination 
of  unbiased  pattern  recognition  techniques  with  expert  information  about  the  strengths 
and  weaknesses  associated  with  the  data  acquisition  techniques  and  expert  knowledge  of 
the  geology,  foliage,  and  man-made  features  located  at  the  site  under  scrutiny.  Data  from 
multiple  sensing  modalities  are  utilized  in  an  overall  decision-making  algorithm  that  is 
more  accurate  than  any  individual  sensor  on  its  own.  [2,3]  The  data  fusion  algorithms  are 
tailored  to  the  data  to  extract  the  maximum  information  available  from  wide-area 
assessment  sensing  technologies.  Sensor  data  are  further  leveraged  by  correlation  with 
historical  target  and  expert  site  information  in  the  data  fusion  algorithms.  In  this  manner, 
intelligent  data  fusion  is  able  to  augment  the  strengths  of  the  different  data  types  while 
minimizing  their  individual  weaknesses  to  provide  the  most  accurate  wide-area 
assessment  for  the  site  given  the  available  information. 

To  date,  effective  wide-area  assessment  has  been  hindered  by  a  lack  of  accurate  target 
and  range  information.  In  addition,  no  single  sensing  technology  has  been  both  accurate 
and  cost-effective  in  surveying  entire  sites.  The  data  fusion  framework  under 
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development  in  project  MM- 15 10  is  intended  to  bridge  this  performance  gap  and  to  do  so 
in  a  manner  that  is  as  independent  of  the  site  data  as  possible,  thus  ensuring  it  is  as  widely 
applicable  as  possible.  Therefore  algorithm  development  is  designed  to  emphasize 
methods  for  feature  extraction,  data  fusion,  and  decision  making  that  are  general  in  nature 
and  not  overly  site  dependent.  The  data  fusion  framework  will  provide  a  set  of 
algorithms  and  a  process  by  which  sensor  data  and  auxiliary  information  can  be 
combined  to  enhance  ordnance-related  signal  and  reduce  false  positives,  independent  of 
the  specific  sensor  data  and  information  available  for  any  one  site.  The  initial  studies 
described  in  this  report  are  focused  on  feature  extraction  and  pattern  recognition 
techniques  that  will  provide  input  to  the  data  fusion  algorithms. 

Results  for  Year  One.  Multiple  wide-area  surveys  with  different  sensing  technologies 
have  been  acquired  under  the  ESTCP  Wide-Area  Assessment  Pilot  Program.  The  data 
resulting  from  wide-area  surveys  of  the  Pueblo  Precision  Bombing  Range  (PBR)  #2 
along  with  associated  expert  information  regarding  the  site  were  used  for  this  study.  The 
Naval  Reseach  Laboratory  (NRL)  reviewed  all  the  available  data  obtained  from  the 
ESTCP  WAA  Pilot  Program  survey  of  Pueblo  PBR  #2.  This  included  magnetometry 
data  acquired  via  helicopter  survey  (Helimag),  LiDAR  data  in  two  resolution  scales  from 
airborne  surveys,  and  orthophotography  data  from  airborne  surveys.  Efforts  at  importing 
and  registering  data  of  various  formats  into  the  MATLAB  environment  were  successful. 
Utilities  for  the  importation  of  arbitrary  geo-referenced  image  data  and  text  data  in  XYZ 
format  were  also  constructed  and  tested.  In  addition,  a  displacement  of  approximately 
four  meters  in  geo-reference  information  was  discovered  in  the  orthophotography  data 
acquired  in  2004  relative  to  other  survey  data.  SERDP  and  Skysearch  were  notified  and 
this  discrepancy  was  addressed  and  successfully  rectified. 

NRL  obtained  and  reviewed  auxiliary  information  for  the  Pueblo  PBR  #2.  This  expert 
information  was  comprised  of  topological  maps,  geological  survey  data,  and  the  archive 
search  report  for  the  site.  Historical  usage  indicated  a  training  range  with  a  bombing 
camp  and  nine  precision  bombing  targets.  Anecdotal  information  in  the  archive  search 
report  also  indicated  the  presence  of  a  suspected  75  mm  air-to-ground  target  and  pattern 
gunnery  range.  These  data  were  utilized  to  generate  a  historical  target  feature  map  of 
likely  UXO  contamination.  Geological  survey  data  indicated  that  the  Pueblo  site 
contained  minimal  magnetic  geological  features.  Foliage  and  other  ground  surface 
features  that  may  interfere  with  data  acquisition  were  also  minimal  at  the  site.  Such 
features  are  expected  to  significantly  affect  data  acquisition  and  quality  at  other  sites,  and 
therefore  will  be  incorporated  into  feature  maps  where  relevant. 

NRL  evaluated  the  current  state-of-the-art  in  data  analysis  and  data  fusion  algorithms  for 
the  sensing  technologies  employed  in  the  ESTCP  WAA  Pilot  Program  surveys.  A 
literature  review  indicated  that  the  current  state-of-the-art  was  generally  focused  on 
algorithm  development  for  the  detection  of  specific  objects,  and  was  therefore  more 
suitable  for  local  area  applications  than  for  wide-area  assessment.  Algorithms  for 
orthophotography  data  emphasized  classification  of  ground  cover  type  through  texture 
analyses  or  pattern  recognition.  Though  the  Pueblo  site  lacked  significant  ground  cover, 
these  algorithms  may  be  applied  to  orthographic  data  from  other  surveyed  sites.  For 
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LiDAR,  algorithms  were  specific  to  the  applications,  which  were  mostly  environmental. 
However,  the  general  approach  used  for  algorithm  development  proved  useful  in  feature 
analysis.  For  magnetometry  data,  algorithms  focused  on  physical  modeling  as  a  method 
of  the  detection  of  individual  objects  and  required  sensitivities  not  attainable  in  helimag 
surveys.  Approaches  to  data  fusion  typically  applied  multivariate  analysis  to  time  and/or 
frequency  domain  EMI  data  to  distinguish  individual  objects  as  well.  EMI  is  a  data 
source  not  yet  available  for  wide-area  assessment. 

NRL  performed  feature  selection  specific  to  each  data  source  to  optimize  the  extraction 
of  information  relevant  for  wide-area  assessment.  As  part  of  this  effort,  NRL  also 
developed  and  implemented  pattern  recognition  algorithms  for  all  selected  features  in  the 
sensor  data  obtained  from  the  ESTCP  WAA  Pilot  Program  surveys.  Methods  for  feature 
maps  suitable  for  auxiliary  information  were  discussed  above.  Craters  were  the  principal 
UXO-related  feature  of  the  LiDAR  and  orthophotography  data,  and  total  magnetic 
content  for  the  magnetometry  data.  For  the  LiDAR  and  orthophotography  data,  the  main 
components  of  the  backgrounds  to  feature  selection  were  geology,  foliage,  and  man-made 
structures.  For  the  magnetometry  data,  geology  and  man-made  structures  contributed  to 
the  background  while  foliage  was  not  a  significant  factor. 

Of  the  three  survey  data  sets,  the  orthographic  data  were  the  least  informative  source  for 
the  Pueblo  PBR  #2.  Extraction  of  craters  was  not  nearly  as  efficient  as  with  the  LiDAR 
data,  and  the  minimal  presence  of  foliage  and  man-made  structures  at  this  site  made 
positive  identification  challenging.  Thus,  autonomic  pattern  recognition  yielded  features 
with  very  low  signal-to-noise  ratios.  Crater  extraction  from  the  LiDAR  data  was 
successful.  An  algorithm  based  on  the  circular  Hough  transform  was  able  to  extract  a 
value  of  four  meters  as  the  characteristic  diameter  of  craters  at  the  Pueblo  site.  Further,  a 
pattern  recognition  algorithm  based  on  the  morphology  of  the  craters  was  developed  to 
locate  them  in  the  LiDAR  data.  This  information  was  then  converted  into  a  feature  map 
describing  the  density  aspect  of  craters.  Feature  maps  describing  the  intensity  and  quality 
aspects  to  craters  may  also  provide  data  fusion  algorithms  with  additional  discriminatory 
information. 

Feature  selection  was  most  successful  with  the  magnetometry  data.  The  total  magnetic 
signal  approach  quickly  yielded  a  feature  map  that  displayed  all  relevant  ferromagnetic 
signals  from  ordnance-related  material  and  man-made  structures  present  in  the  data  from 
the  surveyed  area  of  the  Pueblo  PBR  #2.  Due  to  the  minimal  geomagnetic  features  at  the 
Pueblo  site,  a  simple  threshold  proved  effective  in  eliminating  geologic  background, 
which  is  expected  to  be  more  significant  at  other  ESTCP  WAA  Pilot  Program  sites.  A 
pattern  recognition  algorithm  was  developed  to  separate  ordnance-related  signal  from  the 
ferromagnetic  background  of  man-made  structures.  This  information  was  then  converted 
to  a  feature  map  describing  the  density  aspect  of  ordnance-related  material.  Separate 
feature  maps  describing  the  intensity  and  morphology  aspects  of  the  ordnance-related  and 
man-made  components  of  data  are  also  expected  to  provide  additional  discriminatory 
information  for  data  fusion. 
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NRL  developed  a  multi-resolution  algorithm  for  converting  feature  maps  to  probability 
densities,  which  are  required  for  the  proposed  Bayesian-based  approach  to  data  fusion. 
The  multi-resolution  algorithm  used  a  computationally-intensive  approach  similar  to  that 
of  Parzen  windows.  [5]  The  generation  of  probability  densities  is  a  key  enabling 
component  for  data  fusion  work  slated  for  year  two. 

NRL  also  explored  methods  and  algorithms  for  locating  feature  correlations  among  the 
various  data  sources.  Visual  inspection  of  data  from  the  three  surveyed  sources  revealed 
apparent  spatial  collocation  of  crater  and  ferromagnetic  features.  To  quantify  these 
correlations,  principal  components  analysis  (PCA)  was  applied  to  the  data  sets 
simultaneously.  The  application  of  PCA  resulted  in  no  useful  information  as  the  features 
were  too  obstructed  by  background.  However,  comparison  of  the  crater  and 
magnetometry  feature  maps  for  the  southern  portion  of  the  Pueblo  site  to  the  combined 
feature  map  indicated  that  the  features  were  not  exactly  spatially  correlated.  The 
incidence  of  cratering  was  most  dense  in  the  region  between  the  two  peaks  of  ordnance- 
related  magnetic  signal.  Therefore,  algorithms  to  quantify  feature  correlations  from  data 
subsequent  to  feature  extraction  will  be  developed  in  the  second  year  of  the  project. 

First  year  efforts  for  feature  extraction  were  successful.  Features  relevant  for  wide-area 
assessment  of  UXO  contamination  were  successfully  obtained  from  the  various  data 
sources.  Taken  individually,  the  results  for  each  data  source  (expert  information, 
orthophotography,  LiDAR,  and  magnetometry)  did  not  provide  an  accurate  wide-area 
assessment  of  the  Pueblo  site.  However,  when  the  results  were  considered  in  tandem  the 
extracted  features  provided  a  more  accurate  wide-area  assessment  of  the  Pueblo  site.  For 
example,  the  crater-like  feature  density  map  filled  in  the  gap  between  the  two  peaks  in  the 
southern  portion  of  the  magnetometer  signal  density  map.  Thus,  the  target  range  known 
to  be  present  in  this  area  was  used  more  extensively  than  indicated  by  historical  sources. 
In  addition,  the  density  maps  and  the  combined  map  all  confirmed  the  presence  of  a 
second,  but  less  used,  target  range  located  in  the  northern  portion  of  the  Pueblo  site.  No 
evidence  in  these  maps  supports  the  existence  of  the  suspected  75  mm  range.  Ground 
truth  dug  in  all  three  of  these  regions  affirmed  these  conclusions. 

Conclusions:  The  first  year  of  project  MM-1510  successfully  demonstrated  the 
feasibility  of  feature  extraction  from  wide-area  assessment  survey  data.  In  contrast  to  the 
individual  data  sources,  feature  extraction  yielded  enhanced  data  for  the  Pueblo  PBR  #2 
that  was  well-suited  for  data  fusion.  Preliminary  combination  of  feature  maps  from  the 
various  data  sources  yielded  a  map  for  the  Pueblo  site  that  was  more  accurate  than  any 
one  data  source  alone.  Probability  densities  were  generated  from  the  feature  maps  and 
make  possible  the  combination  of  estimates  of  data  quality,  UXO-related  features,  non- 
UXO  backgrounds,  and  correlations  among  the  data  sets  in  a  Bayesian-based  approach  to 
data  fusion. 

The  feature  extraction  algorithms  and  the  tiered  approach  to  data  fusion  developed  in  the 
first  year  will  enable  a  data  fusion  framework  for  wide-area  assessment  to  be  an  effective 
tool  for  UXO  remediation  of  other  sites.  Although  tailored  to  the  specific  data  sources, 
the  algorithms  developed  are  general  in  nature  and  not  overly  site  dependent.  Large 
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variations  in  the  backgrounds  for  extracted  features  are  expected  for  different  remediation 
sites.  However,  these  variations  will  most  affect  the  signal-to-noise  ratios  in  the  maps  of 
extracted  features.  The  strength  of  a  data  fusion  approach  is  that  it  provides  resilience  to 
such  variations  by  maximizing  the  information  obtained  from  the  different  data  sources 
and  minimizing  exposure  to  false  positives  and  background  variations.  The  methods  will 
provide  a  means  to  develop  additional  feature  maps  to  quantify  specific  background 
features.  Thus,  the  feature  extraction  algorithms  and  tiered  approach  to  data  fusion  will 
be  applicable  to  data  sets  from  other  survey  sites.  In  summary,  the  results  from  the  first 
year  of  project  MM- 15 10  have  only  strengthened  the  case  for  the  effectiveness  of  a  data 
fusion  approach  for  wide-area  assessment. 

Benefits.  Key  advantages  of  systems  based  on  data  fusion  are  the  ability  to  reduce  false 
positives  while  retaining  high  detection  rates  and  the  inherent  flexibility  of  Bayesian 
decision  trees.  Such  a  framework  utilizes  the  information  it  has  available,  without 
necessarily  requiring  that  the  exact  same  inputs  be  present  at  each  application.  This 
flexibility  is  an  important  feature  of  the  data  fusion  approach  as  it  is  expected  that,  for  a 
number  of  reasons,  it  will  rarely  be  the  case  that  exactly  the  same  types  of  data  will  be 
available  for  analysis  each  time  a  wide-area  UXO  assessment  is  performed.  Previous 
work  at  NRL  has  demonstrated  the  success  of  such  techniques  in  simultaneously 
improving  detection  and  reducing  false  positives  in  shipboard  damage  control  system 
architectures  based  around  multiple  sensor  inputs.  It  is  expected  that  these  benefits  will 
be  realized  in  UXO  detection  applications  as  well.  Additional  benefits  that  will  follow 
from  this  research  are  a  better  understanding  of  the  relationships  between  different  sensor 
technologies,  the  impact  of  these  relationships  on  overall  efficacy  at  UXO  detection,  and 
ultimately,  the  knowledge  to  help  design  more  efficient  UXO  surveys  in  the  future. 

Future  Applications.  MATLAB  was  chosen  as  the  computational  platform  for  initial 
algorithms.  As  a  research  tool,  MATLAB  provides  an  extensive  code  base,  with  the 
ability  to  port  MATLAB  implementations  to  other  computational  platforms  following 
development.  It  is  expected  that  MATLAB  will  not  be  utilized  as  a  production  platform 
to  implement  the  data  fusion  framework  for  use  by  site  administrators.  Rather,  a  solution 
that  involves  partnering  with  an  existing  GIS  software  vendor  would  be  preferable.  In 
the  final  year,  we  will  meet  vendors  suggested  by  SERDP  program  managers  to  develop 
a  detailed  plan  for  transitioning  the  data  fusion  framework  to  a  platform  suitable  for 
onsite  implementations.  As  part  of  this  effort,  we  plan  to  demonstrate  the  capabilities  of 
the  data  fusion  framework  at  an  appropriate  site. 
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Objective 


The  objective  of  project  MM-1510  is  to  enhance  assessment  of  buried  unexploded 
ordnance  (UXO)  in  large  geographic  areas  through  data  fusion  of  outputs  of  multiple 
sensing  technologies  with  any  available  expert  knowledge  of  the  sites.  The  result  of  an 
intelligent  combination  of  data  feature  analysis  and  site  information  will  be  wide-area 
maps  of  potentially  contaminated  sites  delineating  areas  of  likely  UXO  contamination 
from  those  that  are  free  of  UXO.  Such  maps  show  density  of  ordnance-related  material, 
which  is  related  to  a  likelihood  of  UXO  contamination.  These  assessments  will  then 
direct  the  acquisition  of  data  using  more  accurate,  sensitive,  and  costly  local-area  UXO 
surveys.  Data  from  local-area  surveys  can  then  be  fed  back  into  the  data  fusion 
framework  and  used  to  further  refine  maps  of  these  regions  and  to  provide  more  detailed 
assessments. 

Project  MM-1510  centers  on  the  development  and  demonstration  of  a  cohesive  data 
management  and  decision  making  utility  for  enhancing  UXO  survey  efforts.  This  will  be 
accomplished  by  providing  an  algorithmic  means  of  utilizing  complementary  information 
from  multiple  sensing  technologies  corroborated  with  expert  site  information  to  reduce 
overall  false  alarm  rates  and  thus  enhance  wide-area  UXO  assessment  efforts.  A  tiered 
approach  to  UXO  assessment  is  being  developed  in  which  data  from  the  wide-area 
sensing  technologies  are  first  processed  by  data  fusion  algorithms.  The  resulting  output 
is  then  used  to  help  direct  progressively  more  local  and  time  intensive  efforts  where  they 
are  most  needed.  The  process  as  a  whole  is  called  the  “data  fusion  framework”  and  will 
benefit  site  remediation  by  allowing  more  efficient  direction  of  time,  labor,  and 
resources. 

This  report  describes  the  first  year  of  a  three-year  effort  to  develop  the  data  fusion 
framework.  The  first  year  objectives  were  to  determine  the  feasibility  of  feature 
extraction  methods  to  provide  complementary  data  from  the  various  detection  sources 
that  will  be  used  in  intelligent  data  fusion. 


Background 


Problem  Statement 

Current  estimates  place  the  total  area  of  possibly  UXO-contaminated  sites  at  10  million 
acres,  with  an  overall  cost  of  remediation  with  current  methods  and  sensing  technologies 
in  the  tens  of  billions  of  dollars.  Studies  have  estimated  that  up  to  80%  of  typical  sites  of 
potential  contamination  are  actually  UXO-free,  indicating  a  need  to  quickly  and  reliably 
scan  large  sites  (on  the  order  of  10,000  acres)  in  order  to  rapidly  identify  regions  that  are 
free  of  UXO  and  regions  that  must  be  subjected  to  more  detailed  and  time-intensive 
examination  and  remediation  with  established  UXO  detection  tools. 

Effective  wide-area  UXO  assessment  is  centered  on  the  capability  to  rapidly  scan  large 
tracts  of  land  and  obtain  relevant,  useful  information  in  the  process.  Two  possible  modes 
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of  enhancement  to  wide-area  UXO  assessment  are,  first,  decreasing  the  false  alarm  rate 
of  current  UXO  sensing  technologies,  and  second,  utilizing  alternate  sensing  technologies 
and  survey  methods  that  scan  larger  areas  or  more  rapidly  cover  large  areas  than  current 
sensing  platforms. 


Current  Technology 

Standard,  ground-based  UXO  sensing  technologies  include  methods  such  as  vehicular- 
mounted  time  and  frequency  domain  electromagnetic  induction  (EMI),  total  field 
magnetometry  (TFM),  and  ground  penetrating  radar  (GPR).  These  UXO  sensors  have 
been  deployed  on  ground-based  platforms  such  as  portable  devices,  push  carts,  and  towed 
sensor  arrays.  Standard  analysis  methods  of  these  types  of  data  are  well  described  in  the 
literature  and  have  been  implemented  with  success.  These  methods  typically  rely  on 
generating  theoretical  sensor  response  models,  or  measuring  pure  responses  of  various 
UXO  items  and  then  comparing  survey  data  to  these  models  in  order  to  make  a  detection 
or  classification.  Typically,  the  ability  of  such  techniques  to  cover  wide  survey  areas  is 
limited,  although  current  SERDP  projects  are  assessing  the  utility  of  ground  survey 
transects  to  increase  this  capability  for  wide-area  applications. 

It  is  expected  that  any  sensing  technology  suitable  for  WAA  will  have  to  be  deployed 
from  fixed  wing  aircraft,  some  other  airborne  platform  such  as  helicopters,  or  even  high- 
altitude  surveillance  such  as  high-resolution  satellite  imagery  in  order  to  rapidly  scan 
larger  areas.  Such  sensing  will  necessarily  be  remote  relative  to  current  methods  for 
detection  and  classification  of  individual  UXO  objects,  and  that  new  sensing  methods  and 
data  analysis  technologies  may  be  more  applicable.  Thus,  increasing  the  area  scanned  is 
likely  to  require  both  new  sensing  technology  and  new  sensor  deployment  platforms. 

Demonstrated  potential  wide-area  UXO  sensing  technologies  fall  into  four  broad  groups: 
electromagnetic  sensing,  ground-penetrating  radar  and  visible  light  imaging,  thermal 
imaging,  and  hyperspectral  imaging,  all  of  which  generate  data  in  the  form  of  maps  (i.e. 
images,  sometimes  with  additional  depth  information)  that  are  spatially  correlated  with 
Global  Positioning  System  (GPS)  data  or  some  other  geo-referencing  tool.  [1]  Of  these, 
electromagnetic  sensors  are  the  most  widely  researched  and  deployed  technology  for  the 
task  of  detection  of  buried  UXO.  The  Naval  Research  Laboratory's  Airborne  Multi- 
Sensor  Towed  Array  Detection  System  (AMT ADS)  is  capable  of  scanning  500  acres  per 
day,  flying  at  a  height  of  three  meters.  However,  at  distances  greater  than  three  meters 
individual  UXO  objects  rapidly  become  lost  among  the  signals  due  to  geological  features; 
a  consequence  of  signal  strength  on  the  cube-root  of  distance.  Nonetheless,  application 
of  magnetometry  is  proving  to  be  useful  for  wide-area  assessment. 

As  the  sensitivity/  ground  penetration  depth  of  direct  UXO  sensing  technologies  drops 
steeply  with  distance,  alternate  techniques  for  wide-area  assessment  will  generally  not 
have  the  benefit  of  sensing  deeply  buried  UXO  objects  directly.  Instead,  they  must  rely 


See  “ Proceedings  of  SPIE:  Detection  and  Remediation  Technologies  for  Mines  and  Minelike  Targets  V 
through  VIII”  and  “ Transactions  on  Geoscience  and  Remote  Sensing,  Vol  2,  No.  3,  (2001)”  for  examples. 
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instead  on  sensing  UXO-related  phenomena  like  spectral  chemical  signatures,  variations 
in  heat  capacity,  and  measurements  of  surface  clutter  and  micro-topological  features. 

The  techniques  of  synthetic  aperture  radar  (SAR),  light  detection  and  ranging  (LiDAR), 
and  high-resolution  aerial  photography  all  yield  information  about  micro-topological 
features  and  surface  clutter  of  wide  areas  under  assessment.  These  sensors  have  the 
benefit  of  functioning  over  much  greater  distances  than  electromagnetic  sensors,  allowing 
them  to  be  deployed  on  fixed  wing  aircraft.  With  this  group  of  sensors,  the  primary  mode 
of  detection  relies  on  the  location  of  micro-topological  features  associated  with  UXO, 
such  as  craters,  infrastructure  remnants,  rather  than  the  direct  detection  of  individual 
UXO  objects.  Thermal  imaging  techniques  yield  information  about  variations  in  heat 
capacity  across  large  areas  and  such  images  have  been  shown  to  be  useful  in  location  of 
UXO  by  sensing  associated  anomalies  in  heat  capacity.  Finally,  hyperspectral  imaging 
techniques  yield  information  regarding  the  chemical  composition  of  wide  areas  of  soil 
that  can  potentially  be  used  to  detect  the  presence  and  concentration  of  UXO.  Towards 
this  end,  six  wide-area  assessment  projects  have  been  completed  under  the  auspices  of 
SERDP  and  ESTCP.  [1]  These  projects  utilized  techniques  that  varied  from  airborne 
infrared  laser  imaging  combined  with  thermal  imaging  (UX-9523),  synthetic  aperture 
radar  (UX-0126,  UX-1070,  and  UX-1 173),  and  airborne  magnetometer  arrays  (UX-0031 
and  UX-3002). 

Data  Fusion  Approach 

To  date,  effective  wide-area  assessment  has  been  hindered  by  a  lack  of  accurate  target 
and  range  information.  In  addition,  no  single  sensing  technology  has  been  both  accurate 
and  cost-effective  in  surveying  entire  sites.  The  data  fusion  framework  under 
development  in  project  MM- 15 10  is  intended  to  bridge  this  performance  gap  and  to  do  so 
in  a  manner  that  is  as  independent  of  the  site  data  as  possible,  thus  ensuring  it  is  as  widely 
applicable  as  possible.  The  algorithm  development  is  therefore  designed  to  emphasize 
methods  for  feature  extraction,  data  fusion,  and  decision-making  that  are  general  in 
nature.  The  data  fusion  framework  will  provide  a  set  of  algorithms  and  a  process  by 
which  sensor  data  and  auxiliary  information  can  be  combined  to  enhance  ordnance- 
related  signal  and  reduce  false  positives,  independent  of  the  specific  sensor  data  and 
information  available  for  any  one  site. 

As  has  been  discussed,  there  are  two  primary  modes  by  which  wide-area  UXO 
assessment  capabilities  can  be  enhanced.  First,  the  high  false  alarm  rate  of  current  UXO 
sensing  technologies  could  be  reduced,  resulting  in  a  reduction  of  time  spent  excavating 
non-UXO  objects  and  allowing  greater  land  coverage  in  a  given  period  of  time.  Second, 
alternate  sensing  technologies  and  survey  schemes  could  be  employed  to  increase  data 
throughput/  land  coverage  in  a  given  period  of  time.  The  data  fusion  approach  under 
investigation  in  project  MM-1510  is  directly  aimed  at  both  of  these  modes  of 
enhancement. 

In  the  context  of  UXO  remediation,  intelligent  data  fusion  is  the  combination  of  unbiased 
pattern  recognition  techniques  with  expert  information  about  the  strengths  and 
weaknesses  associated  with  the  data  acquisition  techniques  and  expert  knowledge  of  the 
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geology,  foliage,  and  man-made  features  located  at  the  site  under  scrutiny.  Data  from 
multiple  sensing  modalities  are  utilized  in  an  overall  decision-making  algorithm  that  is 
more  accurate  than  any  individual  sensor  on  its  own.  [2,3]  The  data  fusion  algorithms  are 
tailored  to  the  data  to  extract  the  maximum  information  available  from  wide-area 
assessment  sensing  technologies.  Sensor  data  are  further  leveraged  by  correlation  with 
historical  target  and  expert  site  information  in  the  data  fusion  algorithms.  In  this  manner, 
intelligent  data  fusion  is  able  to  augment  the  strengths  of  the  different  data  types  while 
minimizing  their  individual  weaknesses  to  provide  the  most  accurate  wide-area 
assessment  for  the  site  given  the  available  information. 

Figure  1  breaks  down  the  proposed  data  fusion  framework  into  five  steps  for  wide-area 
assessment  of  an  individual  site.  The  first  step  is  the  acquisition  of  all  available  sensor 
data  and  auxiliary  site  information.  A  common,  uniform  grid  for  the  site  that 
encompasses  all  surveyed  areas  is  generated  from  a  preliminary  examination  of  the  data. 
Locations  on  the  common  grid  serve  as  indexes  for  database  storage  of  sensor  and  site 
information  pertinent  to  that  grid  location.  The  second  step  is  the  registration  of  sensor 
data  and  site  information  to  the  common  grid  and  their  incorporation  into  an  easily 
accessible  database  format.  As  part  of  the  registration  process,  data  are  analyzed  and  data 
quality  metrics  are  established,  if  possible.  Signals  may  be  interpolated  to  fill  gaps 
between  data  caused  by  geological  impediments  or  from  widely  spaced  “transects”  in 
wide-area  scans. 

The  engine  of  the  data  fusion  framework  is  in  the  third  step,  where  data  enhancement, 
data  fusion,  and  algorithmic  processing  are  performed.  Acquired  sensor  data  are 
processed  according  to  standard  analysis  algorithms  as  well  as  with  additional  feature 
selection  algorithms  specifically  developed  for  each  sensing  technology.  Processed 
sensor  data  and  relevant  auxiliary  information  are  then  combined  with  Bayesian-based 
data  fusion  algorithms  to  uncover  correlations  and  establish  confidence  levels  for 
locations  on  the  grid.  These  algorithms  are  “smart”  in  that  they  incorporate  into  their 
calculations  the  strengths  and  weaknesses  of  the  various  sensor  data  and  pieces  of 
auxiliary  information.  Finally,  the  data  fusion  results  are  analyzed  via  a  Bayesian 
decision  tree  that  examines  grid  locations  across  multiple  size  scales  and  then  calculates 
preliminary  UXO  probabilities  for  the  surveyed  area. 
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In  the  fourth  step,  the  output  of  the  decision  tree  is  assimilated  into  a  wide-area 
assessment  map  of  the  surveyed  area.  Points  on  the  map  are  scaled  and  color  coded  by 
likelihood  to  indicate  the  density  of  ordnance-related  material.  In  step  five,  regions  of  the 
wide-area  assessment  map  delineating  areas  with  a  high  probability  of  UXO 
contamination  can  be  refined  and  enhanced  with  additional  survey  scans  that  acquire 
more  detailed  data.  Data  from  these  more  sensitive  and  higher  resolution  scans  of 
contaminated  areas  can  then  be  fed  back  into  the  data  fusion  framework,  refining  UXO 
probabilities  towards  an  accurate  local  area  assessment  and  dig  sheet.  One  advantage  of 
the  data  fusion  framework  is  that  new  data  can  be  processed  and  incorporated  into  the 
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WAA  map  or  the  refined  local  area  maps  as  it  becomes  available.  This  sequence  (steps  2 
-  5)  is  repeated  until  UXO  contamination  is  acceptably  mapped  for  the  survey  area. 
Remediation  efforts  are  directed  by  a  map-based  display  of  UXO  locations/classifications 
output  by  the  Bayesian  decision  tree  utilized  by  the  data  fusion  framework. 

The  data  fusion  framework  is  based  on  a  tiered  approach  to  data  analysis,  described  in 
steps  1-3  above.  Each  successive  tier  of  analysis  builds  on  the  results  obtained  in  lower 
tiers  by  examining  the  data  in  an  expanding  context  of  new  features  and  auxiliary 
information.  The  development  of  methods  for  each  tier  proceeds  as  follows: 

First,  data  are  acquired  and  registered  to  a  common,  uniform  grid  using  standard 
interpolation  and  averaging  techniques  where  needed.  Data  quality  metrics  are  used  to 
quantify  any  adjustments  made  to  data  and  also  serve  as  a  numerical  confidence  measure 
for  data  at  individual  locations  on  the  grid.  Historical  target,  relevant  geological  data, 
and  other  auxiliary  information  are  also  registered  to  the  common  grid.  Next,  registered 
sensor  data  are  processed  with  standard  signal  processing  tools  and  subjected  to 
exploratory  feature  selection  algorithms  to  identify  UXO-relevant  features.  In  parallel 
with  this,  the  space  of  available  dataset  and  feature  combinations  is  examined  to  identify 
useful  correlations  across  sensor  platforms  and  prune  redundant  information. 
Computationally-tractable  pattern  recognition  techniques  for  selected  features  are  then 
developed  and  implemented.  The  density,  intensity,  and  morphology  of  features  and 
correlations  are  quantified  as  probability  densities  for  all  locations  on  the  grid. 

Next,  a  set  of  data  fusion  algorithms  to  combine  feature  maps  and  auxiliary  information 
are  developed  by  applying  empirical  and  theoretically  guided  heuristics  to  choose 
appropriate  decision  and  data  combination  rules.  The  development  of  data  fusion 
algorithms  is  guided  and  measured  by  their  ability  to  enhance  ordnance-related  signal  and 
reduce  false  positives  in  the  context  of  available  data  and  information.  Finally,  a 
Bayesian  decision  tree  is  developed  to  examine  and  assess  the  results  obtained  from  prior 
analyses  on  multiple  size  scales  in  a  statistically  rigorous  manner.  The  use  of  a  Bayesian 
approach  for  the  decision  tree  provides  resilience  to  missing  or  corrupted  data,  and 
resilience  to  imperfect  feature  classifications  and  auxiliary  information.  Note  that  the 
development  process  is  by  design  not  dependent  on  any  specific  type  of  sensor  data  or 
auxiliary  information.  Data  from  new  sensing  technologies  and  site  information  can 
therefore  be  incorporated  into  the  data  fusion  framework  for  a  site  as  they  are  developed 
and  analyzed. 

Development,  implementation,  and  evaluation  of  project  MM- 15 10  has  been  planned  on 
a  three-year  schedule.  In  the  first  year,  the  principle  tasks  are  to  extract  features  relevant 
for  data  fusion  from  the  sensor  data  and  to  determine  the  feasibility  of  feature  extraction 
to  enhance  ordnance-related  signal  and  reduce  false  positives.  In  the  second  year,  data 
fusion  algorithms  and  the  Bayesian  decision  tree  will  be  developed  and  evaluated.  In  the 
third  year,  the  data  fusion  framework  will  be  optimized  and  demonstrated. 
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Methods 


The  overall  goal  for  the  first  year  of  project  MM1510  was  to  demonstrate  the  feasibility 
of  feature  extraction  from  wide-area  assessment  survey  data  to  enhance  ordnance-related 
signal  and  to  reduce  false  positives  when  used  as  input  to  data  fusion.  Successful 
application  of  data  fusion  requires  two  characteristics  for  input  data  sets.  First,  no  single 
data  source  can  cost-effectively  provide  the  required  information  on  its  own,  and  second, 
the  available  data  sources  must  be  complementary,  providing  additional  information  not 
present  in  the  other  sources.  Without  the  first  characteristic,  there  is  no  need  for  data 
fusion;  without  the  second  characteristic,  there  is  no  benefit  from  data  fusion.  Thus,  first 
year  efforts  were  directed  towards  elucidating  complementary  features  from  the  various 
detection  methods,  geologic  data,  and  historical  site  information.  An  assessment  of 
extracted  features  will  serve  as  the  basis  for  a  go/no  go  decision  for  continuing  the  project 
into  the  second  year:  data  fusion  framework  development.  The  completed  first  year 
components  of  the  data  fusion  framework  are  illustrated  in  Figure  2. 


Figure  2  Diagram  of  completed  first  year  components  of  the  data  fusion  framework. 


The  first  step  was  to  obtain  wide-area  UXO  survey  data  from  the  ESTCP  Wide-Area 
Assessment  Pilot  Program.  Next,  a  review  of  sensor  data  and  acquisition  algorithms  was 
performed  in  order  to  assess  the  feasibility  of  incorporating  current  data  metrics  into  a 
potential  data  fusion  framework.  Concurrently,  available  auxiliary  information  was 
reviewed  for  the  selected  site.  Features  determined  to  be  useful  for  UXO  assessment 
were  identified  in  the  available  data  sets.  Finally,  work  was  performed  to  develop  custom 
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pattern  recognition  and  feature  extraction  algorithms  based  on  our  sensor  data  and  data 
algorithm  review. 

Sensor  Data  Review 

The  first  step  was  the  acquisition  of  data  for  feature  selection  and  algorithm  development. 
This  work  leveraged  data  that  were  acquired  by  SERDP/ESTCP  programs,  specifically, 
data  acquired  during  the  ESTCP  Wide-Area  Assessment  Pilot  Program.  The  WAA  Pilot 
Program  examined  three  former  Department  of  Defense  sites  in  2004  and  2005.  The 
three  sites  surveyed  were:  Pueblo  Precision  Bombing  Range  #2  in  Colorado  (CO), 

Former  Kirtland  Bombing  Targets  N1  and  N3  in  New  Mexico  (NM),  and  the  Military 
Wash  Area  in  the  Borrego  Maneuver  Area  in  California  (CA).  The  principle  sensing 
modalities  utilized  in  each  survey  were  low  altitude  (helicopter)  airborne  magnetometry 
often  referred  to  as  Helimag  and  high-altitude  airborne  LiDAR  and  orthophotography. 

The  next  step  was  the  conversion  and  registration  of  data  sources  to  a  common  grid  and 
format.  MATLAB  [4]  was  chosen  as  the  computational  platform  for  initial  algorithm 
prototyping,  and  this  choice  dictated  the  data  format.  MATLAB  provides  an  extensive 
code  base,  and  the  ability  to  port  MATLAB  implementations  to  other  computational 
platforms  following  development.  It  is  expected  that  MATLAB  will  not  be  utilized  as  a 
production  platform  for  this  work.  For  production  code,  a  solution  that  involves 
partnering  with  an  existing  GIS  software  vendor  would  be  preferable  for  several  reasons. 

Two  grid  spacings  were  selected  for  feature  development,  half  meter  and  one  meter.  It  is 
possible  to  develop  features  in  the  native  space  of  the  data  and  then  register  the  features 
to  a  common  grid.  However,  this  approach  was  not  taken  as  it  does  not  allow  for  an 
examination  of  the  impact  of  data  resolution  on  feature  extraction,  which  may  be  relevant 
to  data  acquisition  costs. 

Expert  Information  Review 

As  part  of  the  Wide-Area  Assessment  Pilot  Program  discussed  earlier,  ground  truth  data 
were  to  be  obtained  for  each  site.  Information  of  this  type  is  useful  for  assessing  the 
utility  of  auxiliary  data,  the  accuracy  of  feature  extraction  from  sensor  data,  and, 
ultimately,  for  evaluating  the  probabilistic  output  the  data  fusion  framework.  Thus,  all 
available  ground  truth  data  were  obtained  for  inclusion  in  feature  development. 

Aside  from  ground  truth  data,  auxiliary  information  regarding  potential  UXO 
contamination  at  a  perspective  UXO  survey  site  was  acquired  to  assess  its  potential  to 
enhance  data  fusion.  Archive  search  reports  for  each  site  consisting  of  items  such  as 
historical  records  of  site  usage,  data  from  previous  surveys,  first  hand  experience  of  local 
inhabitants,  and  other  sources  of  expert  knowledge  were  also  obtained  for  review.  The 
inclusion  of  this  additional  information  is  expected  to  be  useful  in  regions  where 
geological  features  and  surface  phenomena  inhibit  or  alter  sensing  technology  results. 
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Evaluation  of  Current  Sensor  Algorithms 


A  literature  survey  was  performed  in  order  to  research  current  UXO  data  analysis 
strategies  and  to  assess  their  utility  for  extracting  features  for  data  fusion.  Software 
packages  commonly  used  in  the  evaluation  of  UXO  data  were  obtained  and  evaluated  for 
their  ability  to  efficiently  process  large  data  streams,  to  locate  features  relevant  for  wide- 
area  assessment,  and  to  export  such  information  to  the  development  platform.  Data 
provided  in  multiple,  algorithmically-altered  forms  were  evaluated  to  determine  the 
impact  of  the  alterations  to  subsequent  feature  analyses  and  data  fusion. 


Feature  Selection  to  Optimize  Information  and  Pattern  Recognition  Implementation 

The  feature  selection  and  pattern  recognition  tasks  are  closely  linked.  The  investigations 
focused  on  locating  features  relevant  to  UXO  assessment  within  the  wide-area 
assessment  data  gathered,  and  also  on  determining  what  types  of  signal  processing 
algorithms  would  be  useful  in  extracting  those  features  in  an  automatic  fashion.  The 
extracted  feature  sets  will  ultimately  form  the  inputs  into  any  potential  data  fusion 
algorithm.  Utilizing  expert  information  and  ground  truth  regarding  the  survey  site, 
regions  of  interest  were  located.  Data  features  exhibiting  a  causal  or  correlative  link  with 
UXO-related  items  were  flagged  and  examined  as  well  as  features  associated  with  the 
absence  of  UXO-related  items.  The  utility  of  various  data  analysis  techniques  was 
evaluated  for  potential  in  automatic  extraction  of  these  features,  or  otherwise  marking 
regions  associated  with  known  UXO  contamination.  The  acquired  raw  and  processed 
sensor  data  were  also  examined  to  evaluate  the  ability  of  external  signal  processing 
techniques  to  remove  redundancies  and  to  identify  previously  unseen  correlations  and 
features  in  and  between  the  data  sets. 

The  approach  to  feature  analysis  was  based  upon  examining  three  aspects  of  identified 
features,  their  density,  intensity,  and  morphology.  The  density  aspect  of  identified 
features  quantified  the  number  of  features  identified  per  unit  survey  area,  irrespective  of 
feature  strength  or  shape.  The  intensity  aspect  of  features  quantified  their  strength,  and 
the  morphology  aspect  quantified  their  shape.  These  feature  aspects  were  used  to 
construct  probability  density  maps  that  described  the  density,  intensity,  and  morphology 
aspects  of  the  features,  or  combinations  thereof.  Such  maps  are  well-suited  to  a  Bayesian 
approach  to  data  fusion.  Probability  density  was  calculated  via  successive 
approximations  of  aspect  density  (i.e.,  probability  mass)  in  local  regions  of  decreasing 
size,  as  in  the  Parzen  window  approach. [5]  The  goal  was  to  find  features  whose 
probability  density  maps  correlated  (or  anti-correlated)  with  UXO  contamination  as 
identified  with  auxiliary  information  and  human  review  of  the  survey  data.  For  a  variety 
of  reasons,  it  was  not  expected  that,  on  a  feature  by  feature  basis,  all  three  of  these 
aspects  would  necessarily  prove  to  be  simultaneously  useful  to  achieving  this  goal. 

The  final  task  of  the  first  year  of  project  MM1510  was  to  implement  the  feature  selection 
techniques  deemed  to  have  utility  in  extracting  UXO-relevant  features  from  wide-area 
survey  data.  The  last  criterion  for  feature  utility  was  that  its  extraction  from  survey  data 
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be  performed  with  reasonable  computational  resources.  Development  efforts  focused  on 
finding  computationally-tractable  methods  of  executing  the  feature  extraction  algorithms 
on  very  large  data  sets,  and  determining  optimum  parameters  for  algorithm  usage. 


Results  and  Accomplishments 


Sensor  Data  review 

Part  of  the  ESTCP  Wide-Area  Assessment  Pilot  Program  was  assigned  for  use  in  feature 
extraction  development  and  in  training  data  fusion  algorithms.  This  subset  included  the 
data  resulting  from  wide-area  surveys  of  the  Pueblo  Precision  Bombing  Range  #2  (CO) 
and  half  of  the  data  from  Former  Kirtland  Bombing  Targets  N1  and  N3  (Kirtland  AFB, 
NM).  The  remaining  half  of  the  Kirtland  site  and  all  of  the  Military  Wash  Area  in  the 
Borrego  Maneuver  Area  (Victorville,  CA)  site  were  reserved  for  blind  testing  of  the 
resultant  data  fusion  framework.  The  studies  described  here  used  all  the  Pueblo  site 
survey  data  files  and  associated  information. 

The  survey  data  obtained  for  the  Pueblo  site  consisted  of  image  files  containing  aerial 
LiDAR  and  orthophotographic  surveys,  and  a  series  of  text  files  containing  data  from  a 
low-altitude  (helicopter)  survey.  The  aerial  LiDAR  survey  of  the  Pueblo  site  was 
received  as  two  Environmental  Systems  Research  Institute  (ESRI)  “*.adf  ’  files  (arc/info 
binary  file  format).  One  file  comprised  a  square  region  in  the  southwest  comer  of  the 
survey  region  that  had  been  acquired  in  2004.  The  other  file  comprised  the  remaining 
area  of  the  survey  and  had  been  acquired  in  2005.  Within  these  files,  image  pixel  values 
were  represented  as  32-bit  floating  point  numbers.  Images  were  recorded  at  0.4m 
resolution  in  the  first  file  and  at  0.5m  resolution  in  the  second.  The  aerial  orthophoto 
survey  of  the  Pueblo  site  was  received  as  two  Multi-Resolution  Seamless  Image  Database 
(MrSID)  “*.sid”  files.  These  surveys  were  acquired  concurrently  with  the  LiDAR  survey 
and  comprised  the  same  areas  of  the  survey  region.  Each  image  was  recorded  at  a 
resolution  of  0.139m  and  each  pixel’s  value  was  represented  by  three  8-bit  integers 
representing  red,  green,  and  blue  intensities.  The  magnetometry  survey  of  the  Pueblo  site 
was  received  as  a  series  of  ten  ASCII  text  files  containing  six  columns  of  information 
representing  Zone  13  UTM  Northing  and  Easting  coordinates,  magnetometer  reading, 
height  above  ground,  height  above  ellipsoid,  and  time  of  acquisition. 

The  first  step  in  the  review  of  the  acquired  data  was  to  import  the  data  into  the  MATLAB 
computational  environment  for  viewing  and  further  analyses.  A  suite  of  MATLAB 
routines  was  assembled  to  provide  a  turn-key  approach  to  importing  the  image  and  XYZ 
format  data  into  the  MATLAB  workspace  environment.  This  accomplishment  allows  for 
convenient  and  rapid  access  to  all  survey  data  from  within  the  MATLAB  workspace,  and 
makes  possible  the  rapid  importation  of  new  survey  site  data  in  the  future.  The  critical 
functions  that  these  routines  perform  are  as  follows: 
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1 .  Accepting  user  input  of  coordinates  specifying  a  rectangular  region  within 
which  the  survey  site  lies  that  serves  as  a  common  geospatial  grid  of  user 
specified  resolution  upon  which  multiple  survey  results  and  algorithm 
outputs  are  projected. 

2.  Dividing  the  common  grid  region  into  user-specified  square  subsets,  called 
“blocks,”  to  allow  for  analysis  of  local  regions  without  requiring  the 
retrieval  of  data  from  the  entire  survey  site. 

3.  Importation  of  geo-referenced  imagery  into  the  MATLAB  computational 
environment,  and  utilizing  interpolation  or  down  sampling  to  register  it  to 
the  common  grid. 

4.  Importation  of  so-called  “XYZ”  format  data  points  such  as  those  received 
from  the  helicopter  magnetometry  survey  to  the  common  grid. 

5.  Wrapping  all  survey  data  and  associated  metadata  (UTM  coordinates, 
description  of  data  layers,  etc.)  into  a  single  data  structure  (i.e.,  database) 
for  each  subset  block  of  the  common  grid. 


In  order  to  import  geo-referenced  image  data  into  MATLAB,  the  geospatial  data 
abstraction  library  (GDAL)  was  obtained  and  installed  along  with  the  MATLAB 
“mexGDAL”  toolbox  [6],  which  provides  an  interface  between  the  GDAL  library[7  ]and 
MATLAB.  Due  to  the  proprietary  nature  of  the  original  image  file  formats,  the  image 
data  files  were  first  converted  into  geo-referenced  images  in  “GeoTIFF”  format  and  then 
imported  into  MATLAB. 

The  conversion  of  magnetometer  survey  data  into  continuous  surface  estimates  required 
the  following  steps.  For  each  subset  block  of  the  common  grid,  the  magnetometer  data 
text  files  were  automatically  searched  for  data  points  whose  UTM  coordinates  fell  within 
the  geo-referenced  boundaries  of  that  subset  block.  These  points  were  read  into 
MATLAB  as  double  precision  floating  point  numbers.  Next,  the  data  points  falling 
within  a  square  around  a  location  on  the  grid  (i.e.,  pixel)  were  averaged.  The  size  of 
these  grid  pixels  was  specified  by  the  resolution  of  the  common  grid.  Following 
averaging,  the  values  of  pixels  for  which  no  data  points  were  located  were  estimated  via 
interpolation  based  on  the  values  of  neighboring  points.  Lastly,  data  quality  metrics  of 
data  point  density  and  variance  for  each  pixel  on  the  common  grid  surface  were 
calculated  for  diagnostic  purposes  and  for  potential  inclusion  in  future  data  fusion 
algorithms.  These  data  quality  metrics  are  depicted  in  Figure  3(a)  and  (b). 
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Figure  3  Magnetometer  signal  data  quality  metrics:  (a)  data  point  density  and  (b)  variance  per  image  pixel. 


For  the  Pueblo  Precision  Bombing  range  area,  the  common  grid  was  specified  based  on 
an  examination  of  the  furthest  extents  of  each  of  the  different  survey  data  sets.  This  grid 
spanned  the  range  between  UTM  Northing  4,169,390  to  4,178,634  and  UTM  Easting 
614,590  to  618,627,  (Zone  13)  and  represented  a  rectangular  survey  area  of 
approximately  37  square  kilometers.  This  grid  was  then  divided  into  regularly  sized 
blocks  that  were  1024  meters  to  a  side,  starting  at  the  SW  comer  and  moving  south  to 
north  and  west  to  east.  For  each  block,  the  appropriate  data  were  loaded  from  the  source 
file  and  interpolated  to  fit  a  lm  or  0.5m  resolution  grid.  The  data  were  saved  on  a  block 
by  block  basis  in  a  MATLAB  database  where  each  data  layer  occupied  a  different  field  in 
the  structure.  Thus,  correctly  registered,  survey-wide  data  were  made  available  for 
subsequent  feature  extraction  and  processing. 

Note  that  the  survey  area  of  the  Pueblo  site  represented  roughly  40  million  pixels  of 
information  at  one  meter  resolution,  or,  roughly  4  gigabytes  of  information.  For 
comparison,  the  memory  available  in  a  typical  modem  desk-top  computer  is  1  gigabyte; 
in  that  of  scientific  workstation,  4  gigabytes.  As  a  consequence,  an  emphasis  on 
maintaining  computationally-tractable  methods  for  data  analysis  was  a  necessity  for 
algorithm  development. 
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Expert  Information  Review 

The  archive  search  report  and  available  US  geological  survey  information  for  the  Pueblo 
PBR  #2  site  [8]  were  obtained  and  reviewed.  A  detailed  topological  map  showing  man¬ 
made  and  natural  features  within  the  Pueblo  site  was  obtained  for  visualization  and 
analysis  purposes  and  registered  to  the  common  grid  along  with  the  WAA  survey  data. 

The  former  Pueblo  Precision  Bombing  and  Pattern  Gunnery  range  #2  is  in  Otero  County, 
Colorado  and  is  characterized  by  flat  to  rolling  hills  covered  with  grass  and  low  bushes. 
Figure  4  shows  the  portion  of  the  Pueblo  site  that  was  surveyed  with  imagery.  Figure  5 
shows  a  photo  of  the  terrain.  [9]  The  range  is  approximately  20  miles  south  of  La  Junta, 
Colorado.  The  area  was  used  for  cattle  grazing  until  the  War  Department  assumed 
control  of  the  lands  for  a  period  spanning  1942  to  1945.  Currently,  the  lands  are 
primarily  managed  by  the  U.S.  Forest  Service  as  the  Comanche  National  Grasslands  with 
some  portions  leased  to  private  owners  for  grazing  or  else  owned  by  the  State  of 
Colorado.  [10] 
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Figure  4  Pueblo  Precision  Bombing  Range  #2  wide-area  survey  site  within  Packer’s  Gap  1:24,000  USGS 

topographical  map  (7  lA  Minute  Quadrangle  Sheet) 
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Figure  5  Vegetation  typical  of  the  Pueblo  PBR  #2  area.  [9] 


Historical  Usage  and  Suspected  Targets.  The  training  range  consisted  of  a  bombing 
camp  with  two  runways  and  nine  precision  bombing  targets,  a  suspected  75  mm  air-to- 
ground  target,  along  with  an  air-to-ground  pattern  gunnery  range.  A  map  displaying 
these  locations  is  shown  in  Figure  6.  This  knowledge  can  be  utilized  to  generate  a  feature 
map  indicating  high  likelihood  of  UXO  contamination  at  known  target  sites  and  some 
likelihood  at  suspected  target  sites. 


19 


Figure  6  Pueblo  PBR  #2  topographical  map  showing  the  survey  area  (yellow),  man-made  structure 
locations  (blue  dots),  and  known  and  suspected  bombing  target  and  gunning  range  areas  (red). 

Geological  Summary.  The  presence  of  sedimentary  rocks  and  the  lack  of  iron-bearing 
and  magnetic  minerals  [1 1,12]  in  the  study  site  mean  that  minimal  effect,  if  any,  will 
occur  with  the  magnetometer  data.  The  main  geologic  structures  (domes,  synclines)  are 
far  enough  away  from  the  area  to  also  not  affect  the  magnetometer  data.  Any  effects  on 
the  data  will  come  mainly  from  any  man-made  features  in  the  area  (e.g.,  trucks  or  other 
equipment  left  by  surveyors,  electric  lines,  fences,  and  pipelines). 

Although  the  Pueblo  site  contains  minimal  magnetic  geologic  features,  the  following 
effects  on  magnetometer  data  have  to  be  considered  for  future  study  sites:  (1)  magnetic 
ore  bodies  either  exposed  as  an  outcrop  or  beneath  the  surface;  (2)  large  geologic 
structures  containing  magnetic  minerals  (magmatic  intrusions);  (3)  soils  that  contain 
significant  amounts  of  magnetic  iron-bearing  minerals;  (4)  volcanic  rocks  containing 
significant  magnetic  minerals  that  produce  magnetic  anomalies;  (5)  structures  that  occur 
at  tectonic  plate  boundaries  (magnetics  can  be  positive  or  negative)  (6)  fault  zones,  and 
(7)  anticlines  and  synclines  that  effect  the  location  and  concentration  of  rocks  containing 
magnetized  minerals. 
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Physiography,  Relief  and  Drainage.  Otero  County  lies  within  the  physiographic 
province  of  the  Great  Plains.  The  relief  is  gently  undulating.  The  elevation  ranges  from 
3,965  feet  in  the  northeast  to  5,150  feet  on  a  high  mesa  in  the  south-central  part  of  the 
county  for  a  total  relief  of  almost  1,200  feet  across  the  site.  Otero  County  is  drained  by 
the  Arkansas  River  and  its  tributaries.  Major  tributaries  of  the  Arkansas  river  in  Otero 
County  are  the  Apishapa  River,  Timpas  Creek,  Crooked  Arroyo,  Anderson  Arroyo  and 
King  Arroyo.  [13]  The  vegetation  is  typical  of  the  Great  Plains.  The  rolling  surface 
contains  bunch  grass,  sage,  scattered  cactus  and  yucca.  There  are  scattered  low  trees  and 
bushes  in  several  areas.  [14]  The  information  obtained  indicates  that  the  surface 
topography  probed  by  LiDAR  will  be  dominated  in  many  portions  of  the  site  by  washes, 
rolling  hills,  and  bluffs.  A  lack  of  large  vegetation  (e.g.,  trees)  within  the  survey  site 
indicates  that  a  relatively  high  value  can  be  placed  upon  the  acquired  LiDAR  data,  as  the 
extensive  application  of  a  ground  cover  cancellation  algorithm  was  not  required. 

Ground  Truth  Data.  In  the  summer  of  2006,  SERDP  and  ESTCP  personnel  conducted 
a  ground-truth  survey  of  the  Pueblo  site  in  order  to  provide  data  with  which  to  validate 
Wide-Area  Assessment  Pilot  Program  results.  These  data  were  obtained  by  project 
MM1510  researchers  and  is  depicted  in  Figure  7. 
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Figure  7  Pueblo  PBR  #2  topographical  map  with  ground  truth  data.  Red  dots  indicate  locations  of 
ordnance-related,  blue  dots  indicate  non-UXO  scrap,  green  dots  indicate  geologic  feature  (i.e.,  an  empty  dig 
site),  and  magenta  dots  indicate  locations  of  intact  ordnance. 


Evaluation  of  Current  Sensor  Algorithms 

For  orthophotographic  data,  a  wide  variety  of  ground  cover  classification  algorithms  have 
been  reported  in  remote  sensing  literature. [15- 19]  These  algorithms  usually  center  on 
texture  analysis  and  pattern  recognition  schemes  for  discerning  ground  cover  type. 
Current  literature  reports  of  automated  analyses  of  aerial  imagery  tend  to  focus  on  texture 
mapping,  ground  cover  classification  and  detection  of  anomalies.  No  applications 
specific  to  wide-area  assessment  of  UXO  were  reported.  This  result  is  not  surprising  as 
aerial  photography  is  not  capable  of  directly  probing  UXO  objects,  but  must  instead  be 
used  to  locate  visible  features  that  have  a  high  probability  of  co-occurrence  with  UXO. 
With  this  goal  in  mind,  it  is  possible  that  ground  cover  classification  could  be  useful  in 
distinguishing  foliage  and  geologic  features  from  ordnance-related  features. 

LiDAR  survey  data  were  provided  by  Sky  Research  both  as  “raw”  surface  altitude 
measurements  of  the  surveyed  region  and  as  data  processed  with  ground  cover 
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cancellation  algorithms  that  estimate  the  surface  altitude  measurements  as  “bare  earth.” 
Both  versions  of  the  data  were  also  provided  as  shaded  relief  visualizations.  While  the 
shaded  relief  versions  were  advantageous  for  visual  inspection  of  the  data,  they  were  not 
used  for  subsequent  image  analysis  and  feature  extraction  as  the  shading  unnecessarily 
complicated  automatic  numerical  analyses.  This  difficulty  with  shading  is  due  to  the 
choice  of  lighting  angle  in  the  shaded  relief  map.  The  lighting  angle  is  essentially  an 
arbitrarily  chosen  parameter  and  results  in  a  variety  of  different  potential  images  from  the 
same  surface  estimate.  In  contrast,  the  “bare  earth”  estimation  algorithm  employed  by 
Sky  Research  was  effective  at  enhancing  UXO-related  features  as  compared  to 
background  and  was  used  in  subsequent  analyses. 

A  literature  review  was  performed  to  assess  state-of-the-art  sensor  data  algorithms  for 
detection  of  buried  unexploded  ordnance  with  aerial  LiDAR  sensing.  The  review 
revealed  that  such  applications  of  LiDAR  topographical  analysis  are  relatively  new  and 
under-reported  in  the  scientific  literature.  Applications  detailing  various  ground  surface 
estimation  methods  were  predominant  in  the  current  literature,  with  the  subsequent 
topographical  analyses  often  performed  in  a  more  or  less  manual  fashion.  Topographical 
applications  were  generally  divided  between  environmental  monitoring  [20-26]  and 
topographic  mapping  of  urban  areas.  [27-31]  Environmental  monitoring  included  tasks 
such  as  quantitative  assessment  of  beach  and  wetland  erosion,  [20,21]  assessment  of 
forest  and  rangeland  ground  cover,  [22-24]  and  accurate  topographical  mapping  of 
remote  areas,  such  as  underwater  coral  reefs  [25]  or  texture  analysis  of  polar  sea  ice.  [26] 
Urban  mapping  applications  included  tasks  such  as  building,  [27]  bridge,  [28]  and  power 
line  recognition  [29]  and  three-dimensional  modeling  of  man-made  structures.  [30,31] 

Typically,  the  analysis  procedures  and/or  algorithms  utilized  in  each  application  were 
highly  specific  to  the  goals  of  that  application,  and  consequently  not  directly  applicable  to 
wide-area  UXO  assessment.  In  some  cases,  the  use  of  specific  families  of  mathematical 
transforms,  such  as  wavelet  decomposition  [32-34]  and  fractal  dimension  analysis  [35] 
have  been  reported  as  part  of  preliminary  work  to  determine  their  effectiveness  at  feature 
extraction.  In  these  cases,  the  features  examined  and  thus,  the  utility  of  the  approaches, 
were  also  specific  to  the  given  applications.  What  does  become  apparent  in  a  review  of 
current  literature,  however,  is  that  the  general  approach  to  topographical  analysis  of 
LiDAR  data  follows  the  same  general  steps  in  each  case,  regardless  of  application:  First, 
a  set  of  objects  of  interest  was  defined  for  the  task  at  hand.  Next,  the  manner  in  which 
these  objects  uniquely  present  themselves  as  perturbations  in  the  raw  LiDAR  data  cloud, 
or  in  the  LiDAR-generated  surface  was  discerned.  This  knowledge  was  then  used  to 
create  a  feature  extraction  algorithm  that  located  regions  of  the  LiDAR  survey  that  were 
associated  with  the  objects  of  interest.  Although  there  appeared  to  be  no  “turn  key” 
approaches  to  extracting  UXO-related  features  from  LiDAR  survey  data,  an  approach 
that  followed  the  steps  above  while  minding  the  context  of  other  applications  should 
provide  the  best  chance  for  success. 

The  literature  on  helicopter  and  ground  magnetometry  focused  largely  on  individual 
object  detection.  Such  techniques  are  not  necessarily  well-suited  to  the  task  of  wide-area 
assessment.  Typical  electromagnetic  induction  and  total  field  magnetometry  data 
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analysis  algorithms  utilize  physics-based  modeling  to  predict  target  size,  orientation,  and 
depth.  [36-40]  A  number  of  pattern  recognition  and  statistical  techniques,  [37,38,41-43] 
data  fusion  approaches,  [44,45]  and  factor  analysis  algorithms[46,47]  have  been 
investigated  to  discriminate  between  UXO  and  non-UXO  signatures.  Most  applications 
of  multivariate  analysis  involved  time  and/or  frequency  domain  EMI  data,  rather  than 
TFM  such  as  that  acquired  in  the  ESTCP  WAA  Pilot  Program.  Several  tools  that  have 
been  developed  to  assist  in  the  analysis  of  magnetometry  data  are  incorporated  into  a 
module  for  Oasis  Montaj  from  GeoSoft,  Inc.  [48] 

The  utilization  of  physics-based  modeling  tools  is  often  problematic  when  confronted 
with  a  high  density  of  metallic  objects,  metallic  clutter  and  magnetic  geological  features. 
[49]  Therefore,  much  of  this  current  research  focuses  on  the  development  of  algorithms 
and  filters  to  deconvolve  multiple  signals.  While  the  output  of  these  algorithms  may 
prove  useful,  their  goals  are  not  entirely  consistent  with  wide-area  assessment  where  the 
determination  of  areas  of  likely  contamination  takes  precedence  over  identifying 
individual  items  of  UXO.  For  instance,  the  identification  of  ordnance  size  and 
orientation  is  less  important  than  simply  determining  whether  the  signal  is  or  is  not  the 
result  of  UXO-related  scrap.  Thus,  the  application  of  these  algorithms  holds  value  in  so 
much  as  it  may  provide  information  that  assists  in  discriminating  UXO  from  non-UXO 
signatures.  Unfortunately,  due  to  the  immense  size  of  potential  wide-area  survey  sites, 
the  computational  complexity  of  analysis  algorithms  becomes  a  concern  and  the  problem 
must  be  approached  from  this  mindset,  rather  than  from  that  of  a  limited,  local  area 
assessment.  For  this  reason,  the  approach  taken  in  this  work  focused  on  a  morphological 
analysis  of  the  observed  signal  in  TFM  survey  data. 
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Feature  Selection  to  Optimize  Information  and  Pattern  Recognition  Implementation 

Orthophoto  Data  Analysis.  An  examination  of  the  orthophotographic  data  available  for 
the  former  Pueblo  PBR  #2  was  performed  to  ascertain  the  potential  for  automated  feature 
extraction  of  UXO-related  phenomena. 

The  main  features  depicted  by  the  orthophotographic  survey  of  the  site  were  different 
types  of  ground  cover,  evidenced  by  color  and  shape  morphology.  Subjects  such  as  dirt 
roads,  trees,  and  brush  were  plainly  visible.  An  orthophoto  image  at  one -meter  resolution 
of  a  sub  region  of  the  survey  site  is  depicted  in  Figure  8. 
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Figure  8  Orthophoto  of  a  sub  region  of  the  Pueblo  PBR  #2. 

Unfortunately,  none  of  these  features  seemed  to  bear  any  useful  association  with  UXO 
objects.  In  areas  with  heavy  cratering,  it  was  observed  that  vegetation  was  present  within 
the  craters,  although  vegetation  was  also  observed  in  areas  without  cratering,  negating 
this  feature’s  usefulness  for  UXO  indication.  While  the  location  of  man-made  structures 
such  as  fences  and  buildings  was  useful  in  assisting  in  the  interpretation  of  magnetometry 
data,  an  automatic  means  of  doing  so  was  frustrated  by  the  highly  variable  background 
present  in  the  orthophotography  images.  Searching  for  regular  geometric  shapes  (circular 
targets,  fenced  areas,  buildings  and  structures)  was  difficult,  as  edges  were  generally  very 
poorly  defined.  This  was  due  in  part  to  the  relatively  low  contrast  of  the  image  as  well  as 
to  the  variability  of  the  background.  Additionally,  an  examination  of  the  relationship 
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between  color  intensities  on  a  pixel-by-pixel  basis  indicated  a  high  degree  of  correlation, 
shown  in  Figure  9.  This  indicated  that  the  orthophoto  images  contained  very  little 
information  in  terms  of  color. 


red  intensity 


Figure  9  Depiction  of  the  relationship  between  the  three  color  channels  of  the  orthophoto  image  shown  in 

Figure  8  on  a  pixel-by-pixel  basis. 


A  black  and  white  version  of  the  image  in  Figure  8  is  shown  in  Figure  10.  This  image 
was  generated  by  converting  the  8-bit  RGB  image  to  an  8-bit  grayscale  image  and  then 
applying  an  intensity  threshold  of  130  units.  The  map  depicts  areas  within  the  image  that 
are  lighter  than  their  surroundings,  but  does  not  provide  a  path  to  locating  features  of 
interest  for  wide-area  UXO  assessment. 
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Figure  10  Black  and  white  thresholded  version  the  orthophoto  image  in  Figure  8. 

In  an  attempt  to  mitigate  the  effects  of  correlation  between  the  three  color  channels  of  the 
orthophoto  image,  PC  A  [50]  was  used  to  transform  the  original  RGB  space  into 
potentially  more  relevant  descriptors.  The  first  principal  component  of  the  image  shown 
in  Figure  8  is  shown  in  Figure  1 1.  As  can  be  seen,  the  image  is  segmented  into  lighter 
regions  (roads,  bare  earth)  that  are  shaded  blue  in  the  false-color  image  in  Figure  11, 
medium  regions  (grasses),  shaded  green  and  yellow,  and  darker  regions  (dense 
vegetation,  water),  shaded  red.  As  was  expected,  subsequent  principal  components 
provided  negligible  information  due  to  the  high-degree  of  correlation  between  the  color 
channels  of  the  original  image. 
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Figure  11  Image  segmentation  of  the  region  depicted  in  Figure  8  via  principal  components  analysis. 

Unfortunately,  information  obtained  from  ground  cover  classification  methods  provided 
no  clear  path  to  improving  UXO  assessment  in  any  automatic  fashion.  Thus,  the 
potential  utility  of  this  data  set  in  wide  area  UXO  assessment  of  the  Pueblo  PBR  #2  is 
currently  limited  to  assisting  in  visual  assessment  of  the  survey  area.  For  example,  an 
overlay  of  site  imagery  would  assist  in  compiling  expert  information  into  maps  for 
inclusion  into  a  data  fusion  framework.  Potential  applications  remain  in  utilizing  image 
analysis  techniques  to  identify  features  through  classification  of  vegetation  and  ground 
cover,  but  only  if  such  features  are  found  to  be  correlated  with  ordnance-related  objects, 
or,  alternatively,  correlated  with  prominent  features  in  other  survey  data  in  such  a  way  as 
they  assist  in  the  interpretation  of  that  data. 


LiDAR  Data  Analysis.  The  principal  feature  in  LiDAR  data  associated  with  UXO  is 
cratering  of  the  ground  surface.  While  unexploded  ordnance  itself  does  not  cause  crater 
formation,  craters  indicate  a  land  usage  pattern  of  active  ordnance  bombardment,  and 
through  the  assumption  of  collocation,  a  commensurate  increase  in  the  likelihood  that 
UXO  is  present.  The  two  main  criteria  for  a  successful  automatic  crater  detection 
algorithm  are  that  it  be  reasonably  fast  and  highly  selective  for  ordnance-related  craters. 

It  is  important  to  note  that  the  depth  of  the  ordnance-related  craters  observed  is  small 
relative  to  the  overall  changes  in  elevation  experienced  within  the  survey  area,  as  shown 
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in  the  shaded  relief  map  in  Figure  12  and  in  the  top-down  view  shown  in  Figure  13(a). 
This  means  that  the  detection  of  craters  must  occur  through  the  location  of  crater-specific 
arrangements  of  pixel  values  within  the  LiDAR  image  rather  than  directly  through  pixel 
values  themselves.  A  popular  algorithm  used  in  machine  vision  for  locating  objects  of 
radial  symmetry  is  the  circular  Hough  transform.  [60]  The  output  of  the  transform  is  a 
map  of  the  same  dimensions  of  the  input  image,  with  each  pixel  in  the  map  indicating  the 
output  of  an  “accumulator”  that  sums  the  input  data  (or  an  appropriate  transformation 
thereof,  such  as  the  output  of  a  Sobel  gradient  edge  detecting  algorithm)  occupying  a 
constrained  circular  geometry  about  the  pixel.  Map  pixels  with  a  high  accumulator  value 
indicate  areas  at  the  center  of  a  circular  feature,  and  an  appropriate  threshold  is  applied  to 
label  these  features. 


Figure  12  Subsection  of  LiDAR  generated  surface  estimate  for  the  Pueblo  site  showing  ordnance-related 
cratering.  Typical  crater  depths  are  smaller  than  variations  in  surface  elevation. 

Adapting  this  technique  for  crater  detection,  the  initial  LiDAR  surface  data  needed  to  be 
transformed  in  order  to  give  high  signal  value  to  crater  slopes  and  low  value  to  relatively 
flat  areas.  This  was  accomplished  by  estimating  the  gradient  (i.e.  the  slope  of  the 
surface)  at  each  pixel  in  the  LiDAR  image.  Next,  a  moving  window  filter  was  applied 
across  the  gradient  estimate  image,  accumulating  signal  at  a  fixed  radius  range  from  each 
pixel.  This  was  implemented  with  functions  from  MATLAB’s  Image  Processing 
Toolbox.  An  equivalent,  but  faster  executing,  implementation  was  available  on 
MATLAB’s  Central  File  Exchange.  [52]  The  alternate  implementation  used  an  estimate 
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of  the  gradient  direction  as  well,  adding  to  the  accumulator  of  the  pixel  one  radius  unit 
away  in  the  direction  of  the  gradient.  The  approach  is  essentially  the  same  as  that  used 
by  Loy  and  Zelinsky.  [53]  In  either  case,  the  range  of  feature  radii  over  which  to  search 
was  specified.  In  initial  testing,  the  two  circular  Hough  transform  implementations 
examined  provided  essentially  equivalent  output. 

In  order  to  assess  the  circular  Hough  transform’s  ability  to  detect  craters  within  the 
LiDAR  data,  it  was  first  applied  to  the  subset  of  data  depicted  in  Figure  12.  Figure  13(a) 
shows  a  top-down  view  of  the  shaded  relief  map  shown  in  Figure  12.  This  view  is  useful 
for  visualization  purposes,  but  not  for  automated  feature  extraction.  Figure  13(b)  depicts 
a  false-color  map  of  the  raw  LiDAR  surface  data  of  the  same  region.  This  map  indicates 
that  the  dominant  feature  in  this  region  is  a  hillside  sloping  downward  from  the  northwest 
comer  of  the  image  to  the  southeast  comer.  The  craters  located  within  this  region  are 
difficult  to  see,  as  they  represent  a  small  change  in  elevation  relative  to  the  terrain  present 
at  this  location.  A  surface  gradient  estimate  was  calculated  for  every  pixel  in  this  image, 
which  is  shown  in  Figure  13(c).  Here,  the  crater  walls  are  clearly  visible  against  the 
relatively  small  slope  of  the  surrounding  area.  Finally,  Figure  13(d)  depicts  the  output  of 
a  circular  Hough  transform  applied  to  this  data.  Crater  locations  are  clearly  visible  and 
are  selectively  detected  against  other  surface  features,  such  as  small  hills  and  roadways. 
Further,  the  output  of  the  circular  Hough  transform  indicated  that  ordnance-related  craters 
in  this  region  were  relatively  uniform  and  were  typically  four  meters  across. 
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Figure  13  The  process  of  applying  a  circular  Hough  transform  to  locate  craters  in  LiDAR  data.  Figure 
13(a)  is  a  shaded  relief  map  of  a  subsection  of  LiDAR  data  with  ordnance-related  cratering.  Figure  13(b) 
depicts  a  false-color  map  of  the  raw  LiDAR  surface  data,  indicating  that  the  dominant  feature  of  this  data  is 
a  hillside  sloping  downward  from  the  northwest  corner  of  the  image  to  the  southeast  comer.  In  Figure 
13(c)  the  MATLAB-generated  surface  gradient  estimate  is  shown.  Figure  13(d)  depicts  the  output  of  a 

circular  Hough  transform  applied  to  this  data. 


Following  successful  detection  of  craters  in  a  heavily  cratered  region,  the  algorithm  was 
next  applied  to  the  entire  Pueblo  site.  The  circular  Hough  transform  output  for  the  entire 
survey  area  is  shown  in  Figure  14.  As  with  the  sub  regions,  craters  are  accurately  located 
by  the  transform.  Unfortunately,  the  crater  detection  algorithm  also  proved  to  be  less 
selective  than  anticipated.  As  can  be  seen  in  Figure  14,  large  accumulator  values  were 
registered  for  various  non-ordnance  related  geographic  features  such  as  gullies  and  bluffs, 
in  the  upper  central  portion  of  the  region  and  elsewhere.  This  lack  of  selectivity  was 
likely  to  hamper  the  utility  of  the  circular  Hough  transform  output  for  input  into  a  data 
fusion  based  wide-area  assessment  framework. 
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Figure  14  Survey-wide  map  of  circular  Hough  transform  output.  The  circular  Hough  transform 
demonstrates  a  significant  response  to  non-ordnance  related  features  in  the  upper  region  of  the  survey. 

In  response  to  this  challenge,  a  more  restrictive  algorithm  was  implemented  within 
MATLAB.  First,  the  gradient  and  gradient  direction  for  each  pixel  in  the  raw  LiDAR 
surface  were  estimated,  shown  in  Figure  15(a)  and  (b).  This  step  was  carried  out  in  a 
similar  manner  to  that  utilized  by  Bai,  Shen  and  Wang  in  their  radial  symmetry  feature 
finding  algorithm.  [54]  Pixels  with  gradient  magnitudes  less  than  0. 1  units  were  removed 
from  consideration,  resulting  in  the  map  shown  in  pane  (c)  of  Figure  15.  One 
consequence  of  this  approach  is  that  shallower,  less  steeply  sloped  craters  will  be  more 
difficult  to  detect.  The  ability  to  detect  shallower  craters  will  depend  on  the  surface 
texture  of  the  survey  site  and  the  uncertainty  in  the  LiDAR  estimated  surface.  By 
specifying  a  particular  crater  radius  range  and  surface  gradient  threshold,  the  crater 
detection  algorithm  can  be  utilized  to  selectively  search  for  specific  crater  sizes.  Expert 
information  regarding  soil  density  and  depth  as  well  as  historical  information  regarding 
which  types  of  ordnance  were  used  at  the  site  could  conceivably  be  used  to  fine  tune 
these  parameters. 
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Figure  15  Estimated  LiDAR  surface  gradient  (a)  and  direction  (b)  before  and  after  (c)  gradient  magnitude 
threshold  application,  (d)  A  direction  of  0  radians  indicates  a  positive  surface  gradient  estimate  orientation 

of  due  east. 


Next,  a  windowed  filter  was  applied  across  the  gradient  direction  image  that  calculated 
the  difference  between  the  region  local  to  the  center  pixel  of  the  filter  and  an  “ideal 
crater”  gradient  direction  template.  The  crater  template,  shown  in  Figure  16,  was 
generated  by  assuming  a  circular  crater  of  radius  four  meters  and  calculating  the  direction 
of  the  surface  gradient  at  each  pixel  in  the  hypothetical  crater.  Pixels  that  were  in  the 
center  of  the  crater  and  beyond  the  rim  were  disregarded  in  this  difference  calculation. 
Pixels  with  gradients  that  were  within  10  degrees  of  their  counterpart  in  the  crater 
template  were  located,  with  each  one  counting  as  one  vote.  The  filter  generated  the 
number  of  votes  each  potential  crater  location  garnered,  normalized  by  the  total  number 
of  potential  votes.  Figure  17  depicts  the  difference  in  direction  between  the  crater 
template  and  (a),  a  non-crater  region,  and  (b),  a  crater-containing  region. 
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Figure  16  Surface  gradient  direction  template  for  a  crater  of  four  meter  radius  in  a  one  meter  resolution 

image. 
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Figure  17  Residual  between  data  and  ideal  crater  template  for  (a)  a  non-crater  containing  region,  and  (b)  a 

crater  containing  region. 


Pixels  with  a  value  of  less  than  0.30  (only  thirty  percent  of  surrounding  pixels  point 
inward  to  center)  were  discarded  and  the  remaining  pixels  were  flagged  as  crater  center 
locations.  This  method  greatly  enhanced  rejection  of  geographic  features  within  the 
survey  area  when  compared  to  the  circular  Hough  algorithms  examined  first,  as 
demonstrated  by  the  survey- wide  algorithm  output  shown  in  Figure  18(a). 
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Figure  18  A  survey-wide  map  of  the  crater-detection  algorithm  output  is  shown  in  (a)  with  crater  centers 
dilated  to  enhance  visibility.  Shown  in  (b)  is  a  map  of  estimated  probability  density  for  the  crater 

distribution  in  (a). 

The  final  step  in  the  crater  detection  algorithm  was  to  estimate  the  probability  density  of 
the  crater  feature  found  throughout  the  Pueblo  survey  area,  shown  in  Figure  18(b).  The 
information  provided  by  the  custom  crater  detection  algorithm  is  now  suitable  for  input 
into  high-level  data  fusion  algorithms.  Additional  data  quality  metrics  such  as  crater 
depth  and  edge  slope  statistics  can  also  be  easily  calculated  for  data  fusion  purposes. 
These  metrics  may  become  important  in  situations  where  distinguishing  between 
different  types  of  ordnance  usage  and/or  ordnance  and  non-ordnance  sources  of  cratering 
are  desired. 


Magnetometry  Data  Analysis.  The  phenomena  measured  via  airborne  magnetometry 
sensing  are  deviations  in  the  background  magnetic  field  of  the  earth  due  to  the  presence 
of  ferromagnetic  materials  on  or  below  the  surface  of  the  ground.  Ferrous  materials  that 
may  be  present  at  a  survey  site  include  not  only  UXO  and  ordnance-related  scrap  but  also 
a  background  of  non-UXO  related  signal  comprised  of  man-made  features  such  as 
buildings,  fences,  pipelines,  general  metallic  detritus,  and  magnetically  active  geological 
features.  Thus,  the  main  criterion  for  a  successful  detection  algorithm  for  wide-area 
assessment  of  ordnance-related  material  is  the  accurate  separation  and  extraction  of 
UXO-related  signal  from  background  with  computationally-reasonable  methods. 
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Typically,  analysis  of  magnetometer  readings  involves  fitting  a  physics-based  model  to 
predict  quantities  such  as  object  depth,  size,  and  orientation.  This  approach  can  be 
complicated  by  highly  overlapped  signals  from  multiple  objects  and  from  excessive 
distance  between  the  object  and  the  sensor.  For  the  purposes  of  wide-area  assessment,  it 
was  decided  that,  rather  than  approaching  the  UXO  recognition  from  a  physics-based 
modeling  approach,  it  would  be  advantageous  to  consider  the  problem  from  an  image 
analysis  standpoint,  examining  the  morphological  characteristics  of  total  magnetic  signal 
distributions  throughout  a  given  survey  site.  This  approach  has  the  potential  to  loosen 
requirements  on  magnetometer  data  gathering  and  to  speed  analysis  times  while 
continuing  to  provide  useful  assessments  of  potential  UXO  contamination  for  input  to  a 
data  fusion  framework. 

During  registration  to  the  site  map  grid,  the  magnetometer  signal  readings  were  converted 
to  absolute  value  to  provide  a  total  magnetometer  signal  at  each  pixel.  These  magnetic 
data  are  shown  in  Figure  19  for  the  Pueblo  site.  This  total  magnetic  signal  was  then 
examined  to  look  for  features  that  would  separate  ordnance-related  signal  from  the 
ferromagnetic  background. 
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Figure  19  Estimated  total  coverage  magnetometry  survey  constructed  from  airborne  magnetometry  data 
acquired  over  the  Pueblo  site.  Depicted  are  (a)  absolute  magnetometer  reading,  and  (b)  absolute 
magnetometer  reading  with  a  threshold  of  10  nT  applied  to  exclude  signal  due  to  magnetic  geology. 
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The  principle  algorithmic  challenge  to  feature  detection  within  the  magnetometry  data  is 
the  similarity  of  the  ordnance-related  signal  to  the  non-ordnance  background,  both  in 
terms  of  feature  intensity  and  spatial  location.  Figure  20  displays  magnetometry  data 
from  the  southern  portion  of  the  Pueblo  site,  illustrating  the  similarities  between 
ordnance-related  signal  associated  with  a  ship  target  berm  (outlined  in  red)  and  non-UXO 
related  ferromagnetic  objects  such  as  fence  lines  and  other  surface  structures.  Figure  21 
shows  a  zoomed  view  of  these  items.  Figure  22  displays  the  magnetometry  data  related 
to  these  features  at  even  finer  detail,  exposing  their  similarities.  As  can  be  seen  in  these 
figures,  a  scatter  cloud  of  semi-circular  clusters  of  pixels  corresponding  to  ordnance- 
related  signal  is  located  near  the  ship  target  berm.  The  morphology  of  the  cloud  exhibits 
circular  symmetry  over  several  size  scales,  but  is  incongruous  with  the  linear  morphology 
of  the  fence  with  which  the  cloud  overlaps.  Therefore,  an  initial  examination  of  the 
magnetic  data  was  performed  with  a  wavelet-based  approach  [55,56],  which  is  typically 
well-suited  to  distinguishing  shape  and  scale  dependent  features. 
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Figure  20  Subsection  of  the  airborne  magnetometry  survey  containing  a  ship  target  (indicated  in  red), 
ordnance-related  scrap,  and  man-made  structures. 
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Figure  21  Enlarged  regions  of  Figure  20  containing  a  ship  target  (indicated  in  red),  ordnance-related  scrap, 

and  a  man-made  structures. 
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Figure  22  Further  enlarged  region  of  Figure  20  showing  the  data  similarity  of  the  scatter  cloud  and  fence 

features. 


Wavelet  Analysis.  The  first  step  was  to  explore  “denoising”  of  the  wavelet 
decomposition  of  the  magnetometry  data.  Denoising  is  an  umbrella  term  in  the  signal 
processing  community  that  refers  to  methods  which  remove  portions  of  the  signal, 
typically  unwanted  noise,  but  which  can  also  be  structured  background  features.  Noise  is 
generally  captured  by  small-valued  wavelets  in  finely  detailed  scales.  Its  removal  was 
expected  to  enhance  the  semi-circular  clusters  of  ordnance-related  signal  in  the  data. 
Large  scale  morphological  features  such  as  the  scatter  cloud  and  the  fence  line  are 
generally  captured  by  large  valued  wavelets  at  all  size  scales  and  were  expected  to  be 
enhanced  after  the  removal  of  the  least  significantly  valued  wavelets.  The  impact  of 
removing  small  valued  wavelets  from  the  finely  and  coarsely  detailed  scales  in  the 
wavelet  decomposition  of  the  magnetometry  data  was  examined  after  applying  a  standard 
wavelet  denoising  algorithm.  [57,58]  Even  though  several  wavelet  families  and  widely 
varying  noise  thresholds  were  applied,  denoising  of  the  magnetometry  data  did  not 
enhance  ordnance-related  signal  beyond  that  achievable  by  simply  thresholding  the 
magnetometry  data  at  an  intensity  of  ten  units  (i.e.,  keeping  data  with  values  greater  than 
or  equal  to  10  nT).  The  threshold-applied  magnetometry  data  are  shown  in  Figure  19(b). 
Close  comparison  with  Figure  19(a)  shows  that  thresholding  was  effective  in  eliminating 
small  valued  and  unstructured  ferromagnetic  background  distributed  fairly  uniformly  in 
the  magnetometry  data  across  the  Pueblo  site. 
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Though  denoising  was  not  effective  at  enhancing  magnetometry  features,  it  did  reveal 
that  the  magnetometry  data  were  of  good  quality  in  the  sense  that  was  relatively  noise- 
free.  Pixel  intensities  greater  in  magnitude  than  approximately  one  unit  appeared  to 
consistently  correspond  to  ferromagnetic  objects  on  or  below  the  surface  of  the  ground, 
rather  than  to  sources  of  false  positives  specific  to  the  magnetometry  sensor  system.  By 
association,  features  in  the  magnetometry  data  should  also  be  of  similar  good  quality. 

The  next  step  in  analyzing  the  wavelet  decomposition  of  the  magnetometry  data  was  to 
look  for  morphological  parameters  that  might  characterize  and  distinguish  the  ordnance- 
related  signal  from  the  ferromagnetic  background.  Because  the  wavelet  decomposition  is 
multi-scale  in  nature,  it  is  able  to  separate  features  by  size.  Therefore,  the  decomposition 
of  the  magnetometry  data  was  examined  to  look  for  morphological  features  at  size  scales 
that  might  correspond  to  ordnance-related  signal  or  ferromagnetic  background,  for 
example,  a  characteristic  diameter  for  the  semi-circular  pixel  clusters 


Figure  23  Example  survey  regions  (a)  with  ordnance-related  signal  and  (b)  free  of  ordnance-related  signal. 
For  each  region  the  corresponding  standard  deviation,  mean,  median  values  of  wavelet  coefficients  per 

scale  (0  =  coarsest)  are  depicted. 


Figure  23  shows  a  comparison  of  two  small  sections  of  the  magnetometry  data.  The 
upper  left  pane  (Figure  23(a))  shows  a  region  replete  with  ordnance-related  signal;  the 
upper  right  pane  (Figure  23(b))  shows  a  region  nearly  free  of  signal  and  ferromagnetic 
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background.  The  lower  portion  of  Figures  23(a)  and  (b)  shows  plots  the  mean,  median, 
and  standard  deviations  of  the  wavelet  decomposition  per  scale  for  the  magnetometry 
data  (a)  and  (b),  respectively.  Although  each  wavelet  statistic  curve  exhibits  a  peak  for 
the  ordnance-related  data,  the  peaks  are  rather  broad  and  are  not  consistent  with  a  single 
scale.  Further,  the  overall  profiles  of  the  wavelet  curves  for  the  ordnance-related  data  are 
very  similar  to  those  of  the  background  image,  differing  significantly  only  in  magnitude. 
Thus,  the  wavelet  decomposition  analysis  concluded  that  the  semi-circular  clusters  of 
pixels  representing  ordnance-related  signal  in  the  magnetometry  data  typically  comprised 
several  size  scales  simultaneously  and  were  so  sparsely  distributed  that  a  better  separator 
of  signal  from  background  was  simply  the  pixel  magnitude.  This  is  also  consistent  with 
the  earlier  denoising  results,  that  is,  the  lack  of  enhancement  to  magnetometry  features. 

Pixel  Clustering  Analysis.  The  next  step  in  the  analysis  of  magnetometry  data  focused 
on  extracting  features  relevant  for  ordnance-related  signal  in  the  context  of  density, 
intensity,  and  morphology  aspects  to  the  data.  Based  on  the  results  from  the  wavelet 
decomposition  analysis,  the  magnetometry  data  were  thresholded  to  ten  units  for  this 
effort  to  remove  spurious  and  small-valued  background.  An  algorithm  was  developed  to 
identify  clusters  of  adjacent  and  above-threshold  pixels  and  group  them  into  “pixel 
islands.”  Adjacency  was  defined  as  the  pixels  eight  nearest  neighbors.  An  island  was 
defined  as  any  cluster  of  two  or  more  adjacent  and  above-threshold  pixels.  Pixels 
meeting  these  criteria  are  shown  in  Figure  24(a)  for  the  Pueblo  site.  Figure  24(b) 
displays  a  preliminary  probability  density  obtained  from  the  density  of  pixel  islands  in 
pane  (a).  A  zoomed  view  displaying  the  pixel  island  components  to  the  ordnance-related 
cloud  and  fence  features  from  Figure  22  are  shown  in  Figure  25(a).  The  binary  version 
used  to  generate  a  probability  density  distribution  is  shown  in  Figure  25(b).  Pixel  islands 
are  able  to  capture  ordnance-related  features  in  the  magnetometry  data,  but  significant 
ferromagnetic  background  from  man-made  structures  is  equally  well-captured. 
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Figure  24  Shown  in  (a)  are  pixel  islands  in  the  magnetometry  data.  Figure  24(b)  is  a  map  of  estimated 

probability  density  generated  from  (a). 
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Figure  25  Shown  in  (a)  are  pixel  island  components  to  features  shown  in  Figure  22.  Figure  25(b)  depicts  a 

binary  version  of  (a). 


42 


In  order  to  distinguish  ordnance-related  features  from  structured  ferromagnetic 
background,  the  morphology  of  the  pixel  islands  was  explored.  Filtering  pixel  islands 
based  on  the  number  of  pixels  each  island  contained  was  effective  in  separating  signal 
from  background  features.  For  example,  Figure  26(a)  displays  all  pixel  islands  in  the 
southern  portion  of  the  Pueblo  site  shown  previously  in  Figure  20.  Figure  26(b)  displays 
only  those  pixel  islands  with  two  to  75  member  pixels.  Note  that  only  a  few  islands  from 
the  man-made  structures  remain.  Panes  (c)  and  (d)  display  pixel  islands  with  75  -  200 
and  200  -  1000  member  pixels,  respectively.  In  these  ranges,  nearly  all  pixel  islands 
belong  to  man-made  structures.  Only  a  few  islands  in  the  densest  regions  of  the 
ordnance-related  scatter  cloud  are  present  in  these  ranges.  Figure  27(a)  displays  pixel 
islands  with  less  than  100  member  pixels  for  the  Pueblo  site.  Figure  27(b)  shows  the 
corresponding  preliminary  probability  density  for  this  pixel  island  feature  range.  The 
scatter  clouds  of  ordnance-related  signal  associated  with  targets  in  the  northern  and 
southern  sections  are  clearly  the  dominant  features.  For  comparison,  results  for  pixel 
islands  with  more  than  100  member  pixels  are  displayed  Figure  28.  Although  the  densest 
regions  of  the  scatter  clouds  are  visible,  the  features  in  Figure  28  are  almost  entirely  due 
to  man-made  structures,  with  the  fence  line  crossing  the  southern  section  clearly  visible. 
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Figure  26  Filtering  of  pixel  islands:  (a)  all  pixel  islands,  (b)  islands  with  2  -  75,  (c)  75  -  200,  and  (d)  200 

-  1000  member  pixels. 
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Figure  27  Shown  in  (a)  are  pixel  islands  containing  less  than  100  member  pixels.  Shown  in  (b)  is  a  map  of 

estimated  probability  density  generated  from  (a). 
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Figure  28  Shown  in  (a)  are  pixel  islands  containing  more  than  100  member  pixels.  Shown  in  (b)  is  a  map 

of  estimated  probability  density  generated  from  (a). 


The  pixel  island  features  contain  additional  information  about  the  magnetometry  data 
making  possible  further  separation  of  ordnance-related  signal  from  structured 
ferromagnetic  background.  First,  the  intensities  of  member  pixels  seemed  to  be  larger  for 
man-made  structures  than  for  ordnance.  This  could  have  been  due  to  the  location  of 
structures  on  the  ground  surface,  appreciably  closer  to  helicopter  than  buried  ordnance,  as 
well  as  to  more  concentrated  ferromagnetic  content,  typical  of  fence  posts  and  pipelines. 
Figure  29  shows  average  pixel  intensities  for  each  pixel  island  in  a  sub  region  near  the 
target  in  the  southern  region  of  the  Pueblo  site.  Intensities  in  the  fence  feature  were  much 
higher  than  those  in  the  dense  cloud  of  UXO-related  scrap  around  the  bombing  target. 
This  approach  may  provide  a  probabilistic  estimate  to  determine  if  features  in  the 
magnetometry  data  are  on  or  below  the  surface  of  the  ground. 
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Figure  29  Average  pixel  island  intensities,  capped  at  55  nT,  for  island  pixels  surrounding  the  target  in  the 
southern  portion  of  the  Pueblo  site.  Surface  clutter  objects  such  as  fence  lines  have  significantly  greater 

intensity  per  pixel. 

Figures  30  and  3 1  depict  the  distribution  of  pixel  islands  in  the  airborne 
magnetometer  data  with  average  signal  intensities  below  and  above  a  threshold  of 
50  nT,  respectively.  As  with  filtering  based  on  pixel  island  size,  good 
discrimination  was  observed  between  signal  associated  with  buried  UXO-related 
scrap  and  that  associated  with  metallic  objects  on  the  surface. 
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Figure  30  Shown  in  (a)  are  magnetometer  signal  pixel  islands  with  average  intensity  less  than  50  nT. 
Shown  in  (b)  is  a  map  of  estimated  probability  density  generated  from  the  pixel  islands  in  (a). 
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Figure  31  Shown  in  (a)  are  magnetometer  signal  pixel  islands  with  average  intensity  more  than  50  nT. 
Shown  in  (b)  is  a  map  of  estimated  probability  density  generated  from  the  pixel  islands  in  (a). 
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Second,  the  shapes  of  pixel  islands  associated  with  man-made  structures  such  as  fence 
lines  were  generally  more  elliptical  than  those  in  the  ordnance-related  scatter  cloud. 
Figure  32,  reproduced  here  from  Figure  25(b),  shows  a  binary  image  of  the  pixel  islands 
shown  in  Figure  22.  Islands  associated  with  the  fence  tend  to  have  a  major  axis  that  is 
much  larger  than  their  minor  axis.  It  may  be  possible  to  build  a  filter  based  on  this 
characteristic  to  help  separate  ordnance-related  signal  from  ferromagnetic  background. 
Taken  as  a  whole,  the  magnetometry  analysis  resulted  in  three  layers  of  extracted  features 
that  are  expected  to  prove  useful  as  inputs  to  the  data  fusion  algorithms  to  be  developed 
in  year  two  of  the  project. 
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Figure  32  Binary  version  of  Figure  22. 


Summary  of  Results  from  Year  One  Tasks 

NRL  reviewed  all  the  available  data  obtained  from  the  ESTCP  WAA  Pilot  Program 
survey  of  Pueblo  PBR  #2.  This  included  magnetometry  data  acquired  via  helicopter 
survey,  LiDAR  data  in  two  resolution  scales  from  airborne  surveys,  and 
orthophotography  data  from  airborne  surveys.  Efforts  at  importing  and  registering  data  of 
various  formats  into  the  MATLAB  environment  were  successful.  Utilities  for  the 
importation  of  arbitrary  geo-referenced  image  data  and  text  data  in  XYZ  format  were  also 
constructed  and  tested.  In  addition,  a  displacement  of  approximately  four  meters  in  geo¬ 
reference  information  was  discovered  in  the  orthophotography  data  acquired  in  2004 
relative  to  other  survey  data.  SERDP  and  Skysearch  were  notified  and  this  discrepancy 
was  addressed  and  successfully  rectified. 

NRL  obtained  and  reviewed  auxiliary  information  for  the  Pueblo  PBR  #2.  This  expert 
information  was  comprised  of  topological  maps,  geological  survey  data,  and  the  archive 
search  report.  Historical  usage  indicated  a  training  range  with  a  bombing  camp  and  nine 
precision  bombing  targets.  Anecdotal  information  in  the  archive  search  report  also 
indicated  the  presence  of  a  suspected  75  mm  air-to-ground  target  and  pattern  gunnery 
range.  These  data  were  utilized  to  generate  a  historical  target  feature  map  of  likely  UXO 
contamination.  Geological  survey  data  indicated  that  the  Pueblo  site  contained  minimal 
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magnetic  geological  features.  Foliage  and  other  ground  surface  features  that  may 
interfere  with  data  acquisition  were  also  minimal  at  the  site.  Such  features  are  expected 
to  significantly  affect  data  acquisition  and  quality  at  other  sites,  and  therefore  will  be 
incorporated  into  feature  maps  where  relevant. 

NRL  evaluated  the  current  state-of-the-art  in  data  analysis  and  data  fusion  algorithms  for 
the  sensing  technologies  employed  in  the  ESTCP  WAA  Pilot  Program  surveys.  A 
literature  review  indicated  that  the  current  state-of-the-art  was  generally  focused  on 
algorithm  development  for  the  detection  of  specific  objects,  and  was  therefore  more 
suitable  for  local  area  applications  than  for  wide-area  assessment.  Algorithms  for 
orthophotography  data  emphasized  classification  of  ground  cover  type  through  texture 
analyses  or  pattern  recognition.  Though  the  Pueblo  site  lacked  significant  ground  cover, 
these  algorithms  may  be  applied  to  orthographic  data  from  other  surveyed  sites.  For 
LiDAR,  algorithms  were  specific  to  the  applications,  which  were  mostly  environmental. 
However,  the  general  approach  used  for  algorithm  development  proved  useful  in  feature 
analysis.  For  magnetometry  data,  algorithms  focused  on  physical  modeling  as  a  method 
of  the  detection  of  individual  objects  and  required  sensitivities  not  attainable  in  helimag 
surveys.  Approaches  to  data  fusion  typically  applied  multivariate  analysis  to  time  and/or 
frequency  domain  EMI  data  to  distinguish  individual  objects  as  well.  EMI  is  a  data 
source  not  yet  available  for  wide-area  assessment. 

NRL  performed  feature  selection  specific  to  each  data  source  to  optimize  the  extraction 
of  information  relevant  for  wide-area  assessment.  As  part  of  this  effort,  NRL  also 
developed  and  implemented  pattern  recognition  algorithms  for  all  selected  features  in  the 
sensor  data  obtained  from  the  ESTCP  WAA  Pilot  Program  surveys.  Methods  for  feature 
maps  suitable  for  auxiliary  information  were  discussed  above.  Craters  were  the  principal 
UXO-related  feature  of  the  LiDAR  and  orthophotography  data,  and  total  magnetic 
content  for  the  magnetometry  data.  For  the  LiDAR  and  orthophotography  data,  the  main 
components  of  the  backgrounds  to  feature  selection  were  geology,  foliage,  and  man-made 
structures.  For  the  magnetometry  data,  geology  and  man-made  structures  contributed  to 
the  background  while  foliage  was  not  a  significant  factor. 

Of  the  three  survey  data  sets,  the  orthographic  data  were  the  least  informative  source  for 
the  Pueblo  PBR  #2.  Extraction  of  craters  was  not  nearly  as  efficient  as  with  the  LiDAR 
data,  and  the  minimal  presence  of  foliage  and  man-made  structures  at  this  site  made 
positive  identification  challenging.  Thus,  autonomic  pattern  recognition  yielded  features 
with  very  low  signal-to-noise  ratios.  Crater  extraction  from  the  LiDAR  data  was 
successful.  An  algorithm  based  on  the  circular  Hough  transform  was  able  to  extract  a 
value  of  four  meters  as  the  characteristic  diameter  of  craters  at  the  Pueblo  site.  Further,  a 
pattern  recognition  algorithm  based  on  the  morphology  of  the  craters  was  developed  to 
locate  them  in  the  LiDAR  data.  This  information  was  then  converted  into  a  feature  map 
describing  the  density  aspect  of  craters.  Feature  maps  describing  the  intensity  and  quality 
aspects  to  craters  may  also  provide  data  fusion  algorithms  with  additional  discriminatory 
information. 
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Feature  selection  was  most  successful  with  the  magnetometry  data.  The  total  magnetic 
signal  approach  quickly  yielded  a  feature  map  that  displayed  all  relevant  ferromagnetic 
signals  from  ordnance-related  material  and  man-made  structures  present  in  the  data  from 
the  surveyed  area  of  the  Pueblo  PBR  #2.  Due  to  the  minimal  geomagnetic  features  at  the 
Pueblo  site,  a  simple  threshold  proved  effective  in  eliminating  geologic  background, 
which  is  expected  to  be  more  significant  at  other  ESTCP  WAA  Pilot  Program  sites.  A 
pattern  recognition  algorithm  was  developed  to  separate  ordnance-related  signal  from  the 
ferromagnetic  background  of  man-made  structures.  This  information  was  then  converted 
to  a  feature  map  describing  the  density  aspect  of  ordnance-related  material.  Separate 
feature  maps  describing  the  intensity  and  morphology  aspects  of  the  ordnance-related  and 
man-made  components  of  data  are  also  expected  to  provide  additional  discriminatory 
information  for  data  fusion. 

NRL  developed  a  multi-resolution  algorithm  for  converting  feature  maps  to  probability 
densities,  which  are  required  for  the  proposed  Bayesian-based  approach  to  data  fusion. 
The  multi-resolution  algorithm  used  a  computationally-intensive  approach  similar  to  that 
of  Parzen  windows.  A  less  computationally  intensive,  but  correspondingly  less  accurate, 
alternative  based  on  two-dimensional  histograms  of  the  data  was  used  to  generate  the 
figures  of  probability  densities  shown  above.  The  generation  of  probability  densities  is  a 
key  enabling  component  for  data  fusion  work  slated  for  year  two. 

NRL  also  explored  methods  and  algorithms  for  locating  feature  correlations  among  the 
various  data  sources.  Visual  inspection  of  data  from  the  three  surveyed  sources  revealed 
apparent  spatial  collocation  of  crater  and  ferromagnetic  features.  To  quantify  these 
correlations,  principal  components  analysis  was  applied  to  the  data  sets  simultaneously. 
The  application  of  PCA  resulted  in  no  useful  information  as  the  features  were  too 
obstructed  by  background.  However,  comparison  of  the  crater  and  magnetometry  feature 
maps  for  the  southern  portion  of  the  Pueblo  site  to  the  combined  feature  map,  all  shown 
in  Figure  33,  indicated  that  the  features  were  not  exactly  spatially  correlated.  The 
incidence  of  cratering  was  most  dense  in  the  region  between  the  two  peaks  of  ordnance- 
related  magnetic  signal.  Therefore,  algorithms  to  quantify  feature  correlations  from  data 
subsequent  to  feature  extraction  will  be  developed  in  the  second  year  of  the  project. 

NRL  outlined  a  tiered  approach  in  the  proposed  data  fusion  framework  for  accomplishing 
data  fusion,  the  principle  task  for  year  two  of  the  program.  Feature  maps  and  their 
conversion  to  probability  densities  suitable  for  a  Bayesian  approach  to  data  fusion  were 
successfully  obtained  from  the  Pueblo  PBR  #2  survey  data.  These  are  a  key  enabling 
component  for  the  data  fusion  framework. 

First  year  efforts  for  project  MM-1510  are  summarized  in  Figure  33.  Features  relevant 
for  wide  area  assessment  of  UXO  contamination  were  successfully  obtained  from  the 
various  data  sources.  Taken  individually,  the  results  for  each  data  source  (expert 
information,  orthophotography,  LiDAR,  and  magnetometry)  did  not  provide  an  accurate 
wide-area  assessment  of  the  Pueblo  site.  When  the  results  were  considered  in  tandem,  for 
example,  as  shown  combined  feature  map  of  Figure  33,  the  extracted  features  provided  a 
more  accurate  wide-area  assessment  of  the  Pueblo  site.  For  example,  the  crater-like 
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feature  density  map  filled  in  the  gap  between  the  two  peaks  in  the  southern  portion  of  the 
magnetometer  signal  density  map.  Thus,  the  target  range  known  to  be  present  in  this  area 
was  used  more  extensively  than  indicated  by  historical  sources.  In  addition,  the  density 
maps  and  the  combined  map  all  confirmed  the  presence  of  a  second,  but  less  used,  target 
range  located  in  the  northern  portion  of  the  Pueblo  site.  No  evidence  in  these  maps 
supports  the  existence  of  the  suspected  75  mm  range.  Ground  truth  dug  in  all  three  of 
these  regions  affirmed  these  conclusions,  as  shown  in  Figure  33. 
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Figure  33  Summary  of  the  first-year  efforts  of  Project  MM- 15 10 
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Summary 


The  first  year  of  project  MM-1510  successfully  demonstrated  the  feasibility  of  feature 
extraction  from  wide-area  assessment  survey  data.  In  contrast  to  the  individual  data 
sources,  feature  extraction  yielded  enhanced  data  for  the  Pueblo  PBR  #2  that  was  well- 
suited  for  data  fusion.  Preliminary  combination  of  feature  maps  from  the  various  data 
sources  yielded  a  map  for  the  Pueblo  site  that  was  more  accurate  than  any  one  data  source 
alone.  Probability  densities  were  generated  from  the  feature  maps  and  make  possible  the 
combination  of  estimates  of  data  quality,  UXO-related  features,  non-UXO  backgrounds, 
and  correlations  among  the  data  sets  in  a  Bayesian-based  approach  to  data  fusion. 

The  feature  extraction  algorithms  and  the  tiered  approach  to  data  fusion  developed  in  the 
first  year  will  enable  a  data  fusion  framework  for  wide  area  assessment  to  be  an  effective 
tool  for  UXO  remediation  of  other  sites.  Although  tailored  to  the  specific  data  sources, 
the  algorithms  developed  are  general  in  nature  and  not  overly  site  dependent.  Large 
variations  in  the  backgrounds  for  extracted  features  are  expected  for  different  remediation 
sites.  However,  these  variations  will  most  affect  the  signal-to-noise  ratios  in  the  maps  for 
extracted  features.  The  strength  of  a  data  fusion  approach  is  that  it  provides  resilience  to 
such  variations  by  maximizing  the  information  obtained  from  the  different  data  sources 
and  minimizing  exposure  to  false  positives  and  background  variations.  The  methods  will 
provide  a  means  to  develop  additional  feature  maps  to  quantify  specific  background 
features.  Thus,  the  feature  extraction  algorithms  and  tiered  approach  to  data  fusion  will 
be  applicable  to  data  sets  from  other  survey  sites.  In  summary,  the  results  from  the  first 
year  of  project  MM-1510  have  only  strengthened  the  case  for  the  effectiveness  of  a  data 
fusion  approach  for  wide-area  assessment. 
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Appendix  A:  Supporting  Data 


The  common  grid  and  feature  maps  are  on  contained  in  the  attached  DVD. 
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